1 Introduction

The multifrequency global positioning system (GPS) and in general the global navigation satellite systems (GNSS) have become excellent ionospheric sounding systems in the last 20 years. This has been possible by exploiting the very well-known predominant (>99.9%) ionospheric electron content dependence affecting to the transionospheric electromagnetic signals like those of GPS. This dependence is proportional to the integrated electron density and inversely proportional to the squared frequency (see for instance Hernández-Pajares et al. 2011, 2014).

Dual-frequency GNSS measurements provide numerous simultaneous precise ionospheric delays in different directions and regions, and with an unprecedented temporal and spatial resolution (towards a delivery in real time). This fact has been facilitated by the large number of transmitting satellites (typically larger than 20 GNSS satellites in view everywhere at any time) and many permanent networks of multifrequency GNSS receivers. Indeed, permanent networks providing data with rate of up to 1 Hz can be found at worldwide scale (like the International GNSS Service, IGS, network, see Dow et al. 2009), at continental scales (like European EUREF, Bruyninx 2004, USA CORS, Snay and Soler 2008 or Australian CORS networks, Janssen et al. 2010) and at regional scales (like CATNET network, Talaya and Bosch 1999, among many others). In this last category, the very dense and wide area networks of Japan (GEONET, Sagiya 2004, with more than 1000 receivers) and South California (SCGIN, Hudnut et al. 2001, with several hundreds) are remarkable examples. This is due to their extremely high spatial sampling which allow, for instance, to directly view different kinds of ionospheric waves. Examples of this last phenomenon, deserving increasing interest in the scientific community during the last years, are the common medium-scale travelling ionospheric disturbances (MSTIDs, see, for instance, a review in Hernández-Pajares et al. 2012), or circular waves associated with strong earthquakes and/or corresponding tsunamis (see, for instance, Galvan et al. 2012).

In such a context, GPS, which provides the majority of useful observations, will be the baseline for this work. The focus of this paper is on one of most popular ionospheric products in the space geodesy scientific and technological communities: the global ionospheric maps. The GIMs are mostly computed from dual-frequency measurements gathered at global scale (Schaer et al. 1996; Mannucci et al. 1998; Hernández-Pajares et al. 1999). Indeed, the GIMs are being systematically produced and openly provided by the IGS ionosphere working group (IIWG) since 1 June 1998 (see Feltens 2003; Hernández-Pajares et al. 2009). They contain the worldwide distribution of the vertically integrated density (number of free electrons per volume unit) of the ionosphere (i.e. the so-called vertical total electron content, VTEC), with a certain spatial and temporal resolution (usually \(5^{\circ }\times 2.5^{\circ }\) and 15 min to 2 h, respectively). The GIMS are typically estimated from a worldwide selected subset of hundreds of permanent GPS receivers, gathering thousands of dual-frequency GPS observations each time interval (such as 30 s), being accessible with a latency from less than 15 min (real-time experimental GIMs), 1 day (rapid GIMs) and up to 1–2 weeks (final GIMs).

Different estimation techniques have been developed by different ionospheric analysis centres, in particular for the slant-to-vertical mapping, with the general assumption of a common worldwide effective height or, alternatively, a tomographic description (see Hernández-Pajares et al. 1999). Regarding the very important aspect of interpolation, different techniques are used, like the spherical harmonic expansions, Schaer (1999), and combined with generalized trigonometric series functions, Li et al. (2015), splines, Mannucci et al. (1998) and kriging, Orús et al. (2005) (see as well second column of Table 2). This last point is also relevant due to the large areas (such as the oceans, Siberia and most of the Southern Hemisphere) with very few available GPS receivers (see an overall study in Hernández-Pajares et al. 2009).

The accuracy of the different techniques, in particular in the IIWG, has been continuously increasing in the last two decades, thanks to the open and daily-basis independent assessment, comparison, and combination with the corresponding weights in terms of a common IGS GIM. This common GIM is able to provide not only high accuracy, but especially very high reliability (availability and continuity) in all the different combination circumstances. In this regard, it has been crucial to define fair, external and complementing assessment techniques. They have allowed the characterization of the strong and weak points of every GIM at different distances of the permanent receivers (see Hernández-Pajares 2004). Summary and, by the first time, direct comparison of those are the main target of this paper. In this way, we expect to facilitate both, the incorporation of new analysis centres to IIWG contributing to new GIMs to hopefully further improve the combined IGS GIM and in general the provision of a summary of simple and useful ways of performing truly external validations of electron content models.

2 GIM concepts

Let us first summarize some main ideas and suggestions about GIMs, which have been very useful to increase the accuracy of GIMs during the almost 20 years of collaborative work within the IIWG:

  1. 1.

    The big challenge since the beginning in IIWG was that the GIMs should have no lack of availability or “holes”. Then, the ionospheric interpolation techniques become crucial, especially in the large regions with sparse coverage of permanent GPS receivers. In this context, the usage of, for example, a solar-magnetic reference frame (where the electron distribution is most stationary), or accurate interpolation techniques like Kriging (see, for instance, the above-mentioned reference Orús et al. 2005), among other accurate techniques, become important (see Fig. 1).

  2. 2.

    The reliability and accuracy of the combined IGS GIM is mostly based on the fair assessment of the consistency and accuracy of the individual GIMs, provided by different IIWG centres (four during the last 12 years: CODE, ESA-ESOC, JPL and UPC, see Hernández-Pajares et al. 2009).

  3. 3.

    The GIM accuracy shall be assessed from independent electron content data, free from unknown biases, and ideally not taking part in the GIM computation.

  4. 4.

    Moreover, the assessment method should be directly related to the physical quantity (the electron content) provided within the GIMs (for instance this point and the previous one are not fulfilled in Rovira-Garcia et al. 2015).

In this context, we will summarize in the next section two successful GIM-independent assessment procedures, which complement each other well:

  1. 1.

    Taking as reference direct VTEC measurements from dual-frequency altimeters (available over the oceans) on the one hand

  2. 2.

    Taking reference measurements of variations of slant ionospheric delay from permanent GPS receivers which have not been involved in the GIM computation (typically within or closer to continents, large islands and existing GPS networks), on the other hand.

Fig. 1
figure 1

Upper plot it can be seen for day 317 (November 12), 2012, the map representing the GPS receivers contributing to the UPC VTEC GIM “UQRG” (computed with a tomographic-kriging technique following Hernández-Pajares et al. 1999; Orús et al. 2005, magenta squares), the ionospheric pierce points (IPPs) of the corresponding observations from 16h15m to 17h15m (yellow points), the JASON2 altimeter IPPs (blue lines, emphasizing in thick- blue one typical JASON2 pass to be detailed in next Figure) and an example of external permanent receiver (PDEL, red circle) suitable for assessing dSTEC. Bottom plot for the same day 317 (November 12), 2012, the VTEC global distribution is represented in TECU, obtained from the UPC GIM UQRG, corresponding to the selected JASON2 pass (16h45m GPS time)

Full size image

It is important to note that the GIMs computation is then quite different from the computation of regional or either continental scale ionospheric models (hereinafter RIMs), based only on GPS data gathered in regions with a good coverage of GPS receivers (such as Europe or USA). In these scenarios, the accuracy of the ionospheric delay provided by RIMs can be higher, due to the higher density of measurements (see, for instance, Hernández-Pajares et al. 2000; Colombo et al. 2002; Juan et al. 2012).

Fig. 2
figure 2

Histogram representing the distribution of the JASON2 VTEC standard deviation, associated with the sliding window smoothing, for the representative datasets analysed in this study, since 2010–2016 (more than 4,000,000 JASON2 observations analysed)

Full size image
Fig. 3
figure 3

Top plot example of altimeter (JASON-2) VTEC (both raw and smoothed observations) versus GIM (“UQRG” by UPC) VTEC comparison (same day as previous figure, day 317 (November 12), 2012, GPS time 16h24m–17h12m GPS time). Bottom plot dSTEC observed (red) and modelled with UQRG GIM (blue) corresponding to the GIM-external GPS receiver PDEL, located in the top map of Fig. 1, during the same day 317 (November 12), 2012

Full size image

3 GIM assessment methods

Two methods have been successfully used to assess GIMs for almost two decades, and they are still in use today:

  1. 1.

    The comparison with direct VTEC measurements provided by dual-frequency altimeters (taken in slightly different frequency bands than GNSS, and with a typical measurement error standard deviation after smoothing of about 1 TECU, see Fig. 2). This reference VTEC is obtained up to the height of the altimeters (typically +1300 km) which includes the full ionosphere and the most contribution of the plasmaspheric electron content.

    The noise of the altimeter measurements can be significantly reduced by a sliding window (of 16 sec. in the example shown in top plot of Fig. 3, see Fig. 2 for overall distribution of standard deviation after smoothing). Moreover, comparing the altimeter VTEC vs rapid UPC VTEC, the missing altimeter-topside electron content is just a small part of plasmaspheric component (above +1300 km and typically up to few TECUs only). Finally, the well-know altimeter bias excess (see Azpilicueta and Brunini 2009) presents a value of few TECUs only in such a way that it still allows a very clear assessment and comparison of the errors of the different ionospheric models (considering, for instance, daily statistics), typically much larger and systematic . This is the reason why the dual-frequency altimeter measurements provide an excellent and independent source for assessing GNSS-based VTEC models in difficult conditions, overseas and typically far from receivers (see for example Hernández-Pajares 2004; Orús et al. 2005; Hernández-Pajares et al. 2009, 2016). This is illustrated in the example given at the top plot of Fig. 3, which corresponds to a typical JASON2 pass, in this case crossing the two equatorial anomaly peaks (see track in thick-blue at the top plot of Fig. 1). It is evident the good agreement of the GIM VTEC with the altimeter VTEC for the most part of the track (excepting for one of the equatorial anomaly peaks—around latitude +18\(^\circ \)— a most difficult modelling region).

  2. 2.

    The comparison with directly observed STEC variations along a phase-continuous transmitter–receiver arc, provided from GPS receivers with accuracies \(\le \)0.1 TECU,Footnote 1 is another well-behaving and useful electron content modelling assessment test. It has been used since 1998 for ranking, weighting and combining the GIMs of IGS centres in terms of a common IGS GIM, by considering the error in providing the STEC difference in the phase-continuous transmitter–receiver arc among two observations with a similar elevation at the opposite place of the arc (see “self-consistency test” in Orús et al. 2005). In our case, we use a slightly different method (hereinafter dSTEC), which is defined just changing the reference ray in the phase-continuous transmitter–receiver arc (the one with the highest elevation instead of the observation with a similar elevation, see Orús 2005—where it was introduced as “Maximum Elevation test”—see Feltens et al. 2011 and below for more details).

    The dSTEC test complements the altimeter VTEC test (see Table 1): first, dSTEC assesses in slant directions, instead of vertically, like the VTEC-altimeter test. And secondly, dSTEC is available close to GPS receivers, in spite of being far from them, like typically the altimeter VTEC measurements over the oceans. But, as was commented above, it is very important in the dSTEC assessment to use GPS receivers as reference GPS which have not been used by any of the GIMs to be compared, to avoid unclear assessments. Examples of such comparison can be seen in Feltens et al. (2011) and Hernández-Pajares et al. (2016), where 4 and 9 different ionospheric models are compared, respectively.

    Indeed, dSTEC (hereinafter \(\Delta S\) in the equations), is defined as the difference in the given STEC and the STEC at the highest elevation, for each given pair of transmitter and receiver, and for a common arc of measurements. This definition tries to offer a proxy for assessing the behaviour of the modelled STEC, in two different directions and times, because typically the highest elevation ray has lower errors, in particular due to the much less relevance of the mapping function (however the self-consistency test assesses the observed accurate difference in STEC for two rays separated in time and space but with the same elevation, as it was indicated above).

    Table 1 Comparison of main favourable factors (pros) and the unfavourable factors (cons) for VTEC-altimeter and dSTEC GPS data
    Full size table
    Table 2 Recent GIMs assessment vs VTEC-altimeter and vs dSTEC-GPS. It is based on a common set of 21 days with available JASON2 observations, on the one hand, and on dSTEC-GPS, providing the daily RMS over +50 independent GPS receivers
    Full size table

    The good news is that \(\Delta S\) can be directly obtained with almost no-effort from the dual-frequency carrier phase combination \(L_I=L_1-L_2\) of the raw dual-frequency GPS carrier phases (\(L_1\) and \(L_2\)) when no cycle slips happen (typically up to few hours in permanent GPS receivers), following Eq. 1, in which the small term of carrier phase windup of \(L_I\) (less than 0.25 TECUs for ground permanent GPS receivers) is considered corrected (see an example of measured dSTEC values at bottom plot in Fig. 3):

    $$\begin{aligned} \Delta S_o = S_o(t) - S_o(t_\mathrm{E_{\max }}) =\frac{1}{\alpha }\left( L_I(t) - L_I(t_{E_{\max }}) \right) \end{aligned}$$
    (1)

    where \(\alpha =\frac{q^2}{8\pi ^2m_\mathrm{e}\epsilon _0}\simeq 40.3\) in S.I. units, being q and \(m_\mathrm{e}\) the charge and mass of the electron, respectively, and \(\epsilon _0\) the dielectric constant in the vacuum (see, for instance, Hernández-Pajares et al. 2010).

    Then, \(\Delta S\) can be used to compare the performance of approximating the observed value \(\Delta S_o\) in terms of the value provided by each given ionospheric model, \(\Delta S_m\). In this regard, we will focus on \(RMS[\Delta S_o - \Delta S_m]\), comprising a wide interval of elevations in each given arc of data, but clearly below \(E_{\max }\). Indeed, the observed dSTEC is a direct and very accurate measurement of the difference in STEC directly derived from the dual-frequency carrier phases, involving different geometries (elevation angles and regions of the ionosphere) and different times. Then, this is a convenient test, when external receivers are considered, for any ionospheric model, regarding the accuracy of its mapping function, the time dependence and the electron content determination itself.

    In the next section we present a dedicated study to characterize how compatible both tests are, by processing a representative dataset in a worldwide set of 26 GPS receivers mostly placed on islands, i.e. collocated with VTEC-altimeter data.

Fig. 4
figure 4

Selected set of 26 GPS receivers located on islands and coast lines

Full size image

4 Comparison of VTEC-altimeter and dSTEC-GPS assessments

Our purpose is to check the consistency between the assessments of VTEC GIMs by means of VTEC-altimeter and dSTEC-GPS observations, by comparing them when the corresponding measurements are collocated. And in this regard, some important aspects should be considered in this new study:

  1. 1.

    In order to have the clearest picture of the assessment comparison, we should select one of the best available VTEC GIMs, in order to prevent an important GIM error jeopardizing it. We have adopted the rapid UPC GIM (latency of 1 day), with a resolution of 15 min, 5\(^\circ \) and 2.5\(^\circ \) in time, longitude and latitude, respectively. Such GIM (with IGS identification “UQRG”) is computed with the UPC TOMION software, by means of a tomographic and kriging combined technique (see Orús et al. 2005; Hernández-Pajares et al. 1999) and it is behaving in particular better than the official UPC GIM: see Table 2 where a recent combined assessment of GIMs has been separately performed with both VTEC-altimeter and dSTEC-GPS data (more details can be found in Hernández-Pajares et al. 2016).

  2. 2.

    The target of comparing VTEC-altimeter with dSTEC-GPS data brings us to consider GPS receivers on islands: in particular 26 worldwide distributed GPS receivers, have been selected during 102 days evenly distributed from last quarter of 2010 to end of 2016 (see Fig. 4). This period, constrained by the availability of the above-introduced UQRG GIMS, covers the most part of variability of the present solar cycle (see Fig. 5).

  3. 3.

    As the result of a compromise between spatiotemporal colocation and VTEC-altimeter and dSTEC-GPS data availability, the following main requirements have been considered to select the altimeter passes (JASON-2 in our experiment): the maximum difference in longitudes, latitudes and times, among the ionospheric pierce points of the altimeter and slant GPS measurements, should be smaller than 12\(^{\circ }\), 10\(^{\circ }\) (coinciding with latitudinal tick marks interval in top plot of Fig. 3) and 900 seconds, respectively. This means 6407 passes with at least 25 VTEC-JASON2 and dSTEC-GPS collocated observations (see distribution in time and latitude of the overall passes in Fig. 6).

Fig. 5
figure 5

Evolution of solar flux during the period analysed in this study (blue) compared with the overall evolution since 1995

Full size image
Fig. 6
figure 6

Number of JASON2 passes processed during this study, versus time (left-hand plot) and versus latitude (right-hand plot)

Full size image

The assessment comparison is firstly summarized in Fig. 7 with the plot of dSTEC-GPS assessment error and VTEC-altimeter assessment error of UQRG GIM versus time and latitude (left- and right-hand plot, respectively). Both VTEC and dSTEC errors evolves in a compatible way along the analysed period of more than 6 years, showing up higher values in agreement with the maximum phase of the present Solar Cycle (see Fig. 5) and compatible as well with the expected higher ionospheric model error at low latitudes where the equatorial anomaly peaks are located.

Fig. 7
figure 7

UQRG GIM model error for both VTEC and dSTEC versus time (left-hand plot) and versus latitude (right-hand plot)

Full size image

The detailed distribution of the UQRG error RMSs and biases can be found in Figs. 8 and 9, for VTEC-altimeter and dSTEC-GPS (left- and right-hand plots respectively). It can be seen that the most common error RMS values for UQRG and dSTEC are 2 and 0.5 TECUs, respectively. For the biases, the most frequent values are zero or almost zero for dSTEC-GPS, and slightly negative (about -0.5 TECU) for VTEC-altimeter, what is compatible with the well-known few TECUs positive bias of altimeters VTEC calibration, combined with the plasmaspheric electron content above the altimeter (see Azpilicueta and Brunini 2009).

Fig. 8
figure 8

Histogram of the distribution of UQRG GIM model error RMS values, for each collocated GPS (right-hand plot) and JASON2 pass (left-hand plot), with a minimum of 25 observations per pass

Full size image
Fig. 9
figure 9

Histogram of the distribution of UQRG GIM model error bias values, for each collocated GPS (right-hand plot) and JASON2 pass (left-hand plot), with a minimum of 25 observations per pass

Full size image

In order to characterize the strength of a potential direct one-to-one relationship between both GIM errors, dSTEC-GPS error versus VTEC-altimeter error, the Pearson correlation coefficient is provided in Table 3, for different statistical parameters per pass over one receiver, and for different minimum number of observations per pass. It can be seen that only when exigent conditions on the number of minimum observations per pass are imposed, a relatively high Pearson correlation is obtained, greater than 0.5 for relative RMS error (RMSVTECrel) with \(N_{\min }\ge 100\) (reaching up to 0.78 when \(N_{\min }=200\)), and for example for RMS error with \(N_{\min }\ge 225\) (you can see in this case the corresponding one-to-one plot in Fig. 10).

Table 3 Pearson correlation coefficient between several statistical parameters of dSTEC-GPS and VTEC-JASON2 errors of UQRG GIM, for different minimum number of measurements per pass over the GPS receivers
Full size table

We can qualitatively justify the tendency to such linear relationship between dSTEC and VTEC relative errors. Indeed, we can approximate the individual dSTEC error \(\epsilon \left[ \Delta S\right] \) as a linear function of the individual VTEC error \(\epsilon \left[ V\right] \), following the development in Eq. 2. It is based on the high elevation angle of the reference dSTEC observation (the error of the mapping function,Footnote 2 \(\epsilon \left[ M\right] \), is then almost zero), assuming VTEC constancy during each (fast and localized) pass of altimeter on the GPS receiver, neglecting as well the lowest elevation ray mapping function error (the toughest hypothesis in spite of \(E > 15^{\circ }\)).

$$\begin{aligned} \epsilon \left[ \Delta S\right]\equiv & {} \epsilon \left[ S-S(E_{\max })\right] \nonumber \\= & {} \epsilon \left[ M\cdot V\right] -\epsilon \left[ M(E_{\max })\cdot V(E_{\max })\right] \nonumber \\\simeq & {} M\cdot \epsilon \left[ V\right] +\epsilon \left[ M\right] \cdot V-M(E_{\max })\cdot \epsilon \left[ V(E_{\max })\right] \nonumber \\\approx & {} \left( M-M(E_{\max })\right) \cdot \epsilon \left[ V\right] \end{aligned}$$
(2)

From Eq. 2, and the above-mentioned hypothesis, the linear relationship for both error RMS, R, is deduced immediately:

$$\begin{aligned} R\left[ \epsilon \left[ \Delta S\right] \right] = \sqrt{\left<{\left( M-M(E_{\max })\right) }^2\right>}\cdot R\left[ \epsilon \left[ V\right] \right] \end{aligned}$$
(3)

where \(<>\) represents the average along the collocated altimeter and GPS measurements. Similarly, but in a less approximate way:

$$\begin{aligned} \Delta S \approx \left( M-M(E_{\max })\right) \cdot V \end{aligned}$$
(4)
Fig. 10
figure 10

RMS of UQRG dSTEC discrepancy [referred to observed GPS value) vs. RMS of UQRG VTEC discrepancy (vs. JASON2 value) for each one of the collocated passes, and with a minimum number of 225 measurements (the blue line represents the equality of both compared quantities]

Full size image

leading finally to:

$$\begin{aligned} R\left[ \Delta S\right] = \sqrt{\left<{\left( M-M(E_{\max })\right) }^2\right>}\cdot R\left[ V\right] \end{aligned}$$
(5)

and to a tendency to get similar relative error values between dSTEC-GPS and VTEC-altimeter:

$$\begin{aligned} \frac{R\left[ \epsilon \left[ \Delta S\right] \right] }{R\left[ \Delta S\right] } \sim \frac{R\left[ \epsilon \left[ V\right] \right] }{R\left[ V\right] } \end{aligned}$$
(6)

5 Conclusions

The GPS-based GIMs have been improving in quality since the start of their generation in the nineties of the last century, in spite of the difficulty in providing reliable values at the most part of the ionosphere (such as over the oceans and/or South Hemisphere), far from permanent GPS receivers, and mostly based on the interpolation techniques. In this regard, the identification and careful systematic application of scientifically well-founded techniques of assessment, like the usage of direct and independent dual-frequency altimeter VTEC measurements, and the dSTEC from GPS receiver not taking part in the GIM generation, have played a fundamental role in the evolution of the GIM computation strategies, within a scientific and friendly spirit of cooperation, as has been possible in the international GNSS service (IGS) since the foundation of its ionospheric working group in 1998.

In this context, two independent and complementing ionospheric assessing techniques of VTEC GIMs, taking as reference the direct dSTEC-GPS and VTEC-altimeter observations, are, probably for the first time, quantitatively compared.

We have adopted the best performing UPC GIM (UQRG), and we have considered JASON2 altimeter collocated VTEC observations over a set of 26 GPS IGS receivers placed mostly on worldwide islands, during 102 days within almost half solar cycle, the last quarter of year 2010 up to full year 2016.

The UQRG GIM VTEC and dSTEC errors, derived from collocated JASON2 passes and GPS receivers on islands, present the most frequent RMS values of 2 TECU and 0.5 TECU, respectively. And they tend in general to a certain linear relationship, especially for the error relative error and RMS error when a minimum high number of collocated measurements is present. This result is analytically justified in the manuscript relating to the definitions of both GIM VTEC and dSTEC errors.

Finally, it can be confirmed that both complementing and independent assessing techniques, dSTEC-GPS and VTEC-altimeter, successively used in previous works to rank VTEC GIMs, show its quantitative consistency, in coincidence with a simple statistical model, when they are directly compared in collocated scenarios. So the authors strongly recommend its usage (as in a major characterization of ionospheric models we are preparing, involving as well authors from other six analysis centres), when a direct and truly external assessment is required for any electron content model of the ionosphere.