Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Background

Continuous glucose monitoring (CGM) technology provides significant advantages to individuals treated with intensive insulin therapy compared with traditional self-monitoring of blood glucose (SMBG). Unlike SMBG, which measures the current glucose level at a single point in time, CGM presents a constant stream of data (measured every 1–15 min), indicating not only the current interstitial glucose level but also trends in glucose direction and velocity of glucose change. Moreover, CGM systems feature alarms that alert users when glucose is or predicted to be above or below programmed glucose thresholds, which makes the technology clinically useful in identifying postprandial hyperglycemia, nocturnal hypoglycemia, and in assisting individuals with hypoglycemia unawareness [43].

Numerous clinical trials have shown that use of CGM improves glycemic control and reduces hypoglycemia in children and adults with type 1 diabetes [6, 15, 20, 27, 38, 41, 48]. However, the clinical benefit of CGM use is directly related to the frequency of use of the technology. In many studies, significant clinical improvements were seen only in those patients who regularly wore their CGM devices 60–70 % of the time [2, 9, 20, 27, 41, 46–48].

As the technology continues to evolve, use of CGM is emerging as a standard of care for diabetes patients managed with intensive insulin therapy [8, 30, 39, 44]. Additionally, CGM is an integral component of the “artificial pancreas” (AP) systems under development, which utilize continuous subcutaneous insulin infusion (CSII) linked with CGM and an automated control algorithm to communicate between the two components to mimic physiologic insulin delivery.

Despite its demonstrated benefits, adoption and routine utilization of CGM has been relatively slow. A large US registry of individuals with type 1 diabetes showed that only 6 % of individuals used CGM as a regular part of their diabetes management [7]. It is also known that many patients who are started on CGM discontinue it shortly thereafter [42].

Although cost is often identified as a hindrance to CGM use; [4, 26] inadequate sensor accuracy may be a more influential obstacle to widespread adoption of CGM for current use in clinical diabetes management and future use in AP systems [13, 28]. In clinical trials, the number of patients who discontinued the studies due to accuracy or sensor-related issues was significant [20, 26, 41]. Therefore, it is important that both clinicians and patients are able to easily assess the performance of the CGM systems they are considering.

2 CGM Performance Assessment

2.1 Sensor Signal

The effectiveness of CGM devices in measuring current glucose and predicting future glucose levels is dependent upon the accuracy and reliability of the CGM signal. The raw (unfiltered) signal of a CGM sensor is the basis for meaningful data output. A raw signal that exhibits strong noise and artifacts requires robust filter algorithms in order to equalize the signal. The characteristics of noise and artifacts will dictate the type of filter algorithm needed. Regardless of the filter algorithm selected, all filters introduce a time lag in that the filtered signal lags behind the raw signal; the stronger the filter algorithm, the longer the time lag. A raw signal of high quality requires minimal filtering and data processing, resulting in a much cleaner sensor response with little delay between the shown data and the raw signal.

2.2 Reference Methodology

Blood glucose measurement is most commonly used as reference data for the analysis of CGM accuracy. However, the limitations of using this reference method must be considered. It is important to consider that this method is comparing glucose concentrations in two body compartments: blood and interstitial fluid. As demonstrated by Basu and colleagues, [5] who was the first to directly measure the transport of glucose from the vascular compartment to subcutaneous tissue, demonstrated that the mean time to appearance of tracer glucose in the abdominal subcutaneous tissue after intravenous bolus is between 5 and 6 min in the resting, overnight fasted state. Moreover, the glucose transport lag time may vary when the patient is not in the fasting state (e.g., glucose levels are rising or declining). An example of this lag is presented in Fig. 1. As a consequence, CGM and blood glucose points that are paired according to their measurement time stamp may be erroneously paired from the physiological point of view. Additionally, the blood glucose readings are paired with a very small fraction of CGM readings, most CGM data are neglected, and deviations of CGM readings from the blood glucose occurring between the blood glucose measurements are not detected at all.

Fig. 1
figure 1

Example of “time lag” between interstitial and blood glucose levels. There is virtually no time lag when glucose is stable (A), whereas the lag is apparent when glucose levels are changing rapidly (B).

Full size image

2.3 Accuracy and Precision

A common metric for assessment of CGM accuracy is the aggregate mean absolute relative difference (MARD) between all temporally matched sensor data and reference measurements across all subjects of a study. Reported as a percentage, MARD is the average of the absolute error between all CGM values and matched reference values. A small percentage indicates that the CGM readings are close to the reference glucose value, whereas a larger MARD percentage indicates greater discrepancies between the CGM and reference glucose values.

Another metric is the median absolute relative difference, which is also expressed as a percentage. Both the MARD and median absolute relative difference are often reported in CGM accuracy studies [29, 34, 50]; however, some studies [10, 22, 49] report only the median difference, which is misleading because it diminishes the impact of outliers. When there are significant numbers of outliers, accuracy appears to be greater.

Although the MARD is a more stringent metric and is easy to compute and interpret for succinct summarization of CGM properties, certain limitations should be considered when comparing reported findings from different accuracy studies [37]. Because the MARD uses blood glucose readings as a reference, the influence of time lags is introduced and the majority of CGM values are neglected. Additionally, the MARD makes no distinction between positive and negative errors or between systematic and random errors.

It is also important to consider the composition of the study cohort and study setting. Are the study subjects prone to severe and/or frequent fluctuations in glucose? How often and how long are they in the hyperglycemic and hypoglycemic range? All of these factors can significantly influence the calculated MARD. For example, in study subjects with relatively stable glucose, the MARD may show close agreement with blood glucose readings. However, in subjects whose glucose is predominantly in the hypoglycemic range or fluctuating between the hypoglycemic and hyperglycemic states, the MARD would likely be less than desirable. Therefore, it is advisable to conduct separate accuracy evaluations for different glucose concentration ranges (e.g., 40–70 mg/dL) and different rates of glucose change (e.g., stable glucose vs. rapidly changing glucose). Additionally, when comparing the accuracy of two or more different CGM systems, it is advisable to perform head-to-head assessments in which the two systems are running simultaneously in each study patient. This will neutralize the potential impact study cohort or study setting differences.

Although use of the MARD provides easily interpreted information about the accuracy of CGM devices compared with reference blood glucose, its utility is impacted by the limitations previously discussed. However, when used in conjunction with calculation of sensor precision, the limitations are diminished (Table 1).

Precision of the absolute relative difference (PARD) is a metric used to compare glucose readings from two identical CGM sensors working simultaneously in the same patient; a lower percentage indicates greater precision (less variance) [3, 37, 50, 51]. It is calculated according to Eq. (1):

$$\begin{aligned} PARD=100\cdot {}\dfrac{Gluc_{sens1}-Gluc_{sens2}}{Mean(Gluc_{sens1}, Gluc_{sens2})} \end{aligned}$$
(1)

Using this metric, all CGM data are considered for performance evaluation. However, there is no correlation to blood glucose concentration. Nevertheless, the comparison of two concurrent CGM sensors provides additional and complementary insights into CGM sensor properties and performance.

CGM performance studies that present both MARD and PARD at various glucose ranges facilitate a more reliable assessment of a given CGM sensors true accuracy and precision characteristics. Data from a study that assessed the accuracy of three different CGM systems that were worn simultaneously (in duplicate) are provided as an example of how use of MARD and PARD allows for more straightforward comparisons of various CGM systems [21].

Table 1 MARD and PARD at various glucose ranges [21]
Full size table

Although significant differences in accuracy and precision between the devices are readily apparent, the data also highlight the differences when comparing overall accuracy/precision across the full glucose range studied (40–400 mg/dL) and within the hypoglycemic range (40–70 mg/dL). This is important because accuracy and precision in the hypoglycemic range is required for the technology to provide timely warnings of immediate or impending hypoglycemia. Failure to provide reliable warnings can impact user acceptance; fear of hypoglycemia is widely recognized as a major barrier to achieving and sustaining optimal glycemic control [16].

3 State of the Art

Despite advances in technology, CGM devices remain less accurate than capillary blood glucose measurements. Some of these deviations are physiological, such as the time lags that occur when glucose levels are rapidly changing. At other times, CGM sensor data may deviate from measured blood glucose values due to inaccurate calibration of the CGM device, sensor dislodgment, and other artifacts such as local decrease of blood flow [25]. If these unpredictable deviations are of sufficient length or magnitude, they could result in erroneous and potentially harmful patient responses (e.g., insulin under-dosing or over-dosing) [32].

It should be noted that the “accuracy gap” between current CGM devices and blood glucose monitoring systems is shrinking. Whereas current blood glucose monitoring systems that fulfill the ISO 1597:2003 criteria exhibit overall MARD values ranging from 4.9 to 6.8 % [45], a recent study of CGM devices showed MARD values as low as 9.2 % [51].

When assessing CGM performance, it is also important to consider differences between the currently available CGM systems. Figure 2 presents comparisons in accuracy (MARD) from four recent studies [17, 18, 21, 33] between the first generation CGM system (FreeStyle NavigatorTM, Abbott Diabetes, Alameda, USA) and more recent CGM systems from Medtronic (Northridge, USA) and Dexcom (San Diego, USA).

Fig. 2
figure 2

Overall MARD from four head-to-head studies

Full size image

Advances in CGM development have also facilitated significant improvements in accuracy at various rates of glucose change as assessed by MARD. Figure 3 presents data from a head-to-head comparison between a commercially available device (Dexcom G4® PLATINUM Dexcom) and a prototype system (Roche Diagnostics GmbH) [40].

Fig. 3
figure 3

Stratified MARD for different rates of glucose change

Full size image

4 Unresolved Issues

In addition to their continuing efforts to improve the accuracy and precision of CGM devices, developers are challenged to address factors that can impact sensor reliability and performance. These factors include motion (including micromotion within the subcutaneous tissue) and pressure, which can affect tissue physiology (interstitial stresses) and local blood flow around the sensor, resulting in signal disruption and/or erroneous glucose readings [25].

4.1 Transient Sensor Signal Disruption

Numerous experiments and clinical studies have revealed occasional, spurious sensor signal dropouts. When this occurs, the glucose values reported by the CGM device are usually lower than the actual values. An example of this is presented in Fig. 4.

Signal disruption most commonly occurs during sleep, due to individuals lying on top of their sensor. These signal dropouts tend to last approximately 15–30 min. Although nocturnal signal drops are not likely to prompt an erroneous response in patients using standalone CGM devices, resulting “false alarms” may provoke patients to turn off their CGM and/or discontinue use [19]. However, signal disruptions are of great concern within the context of sensor augmented insulin pump use in which insulin administration can be automatically suspended when CGM measure values fall below the programmed low-glucose threshold and (eventually) in closed-loop insulin delivery system (artificial pancreas) use. Signal disruption also poses a significant challenge in artificial pancreas development efforts.

Fig. 4
figure 4

Example of nocturnal signal drop-out

Full size image

4.2 Transient Significant CGM Inaccuracies

Whereas signal disruptions are often the result of acute forces such as pressure which can temporarily impact the sensor signal, chronic forces that trigger the foreign body response, and inflammation around the sensor site can impact sensor performance [25]. These forces can include the impact of motion (shear forces) that occur due to normal physiologic activity (e.g., walking, running) [23] and foreign body response [1, 36], which is influenced by the shape [35], size [31] and surface typography [11] of the implanted sensor. Reporting of erroneous interstitial glucose values are unpredictable and may continue over several hours.

5 Next Steps in CGM Development

Increasing standardization of performance metrics will enable CGM technology developers and patients to better assess the accuracy and precision of future CGM devices. The better understanding of the sensor-to-tissue interface will help developers to improve accuracy. As a result, adoption and sustained use of CGM as standalone devices will likely increase. However, the challenges posed by intermittent signal disruptions and transient inaccuracies must be addressed in order for closed-loop insulin delivery development to move forward.

Development of CGM sensors that incorporate intelligent failure detection algorithms, which may include sensors for activity, pressure, temperature, and heart rate, is a long-range goal in CGM development. The addition of glucose pattern recognition capability linked with global position data would enhance the utility of CGM in the future. However, use of multiple sensors presents a potential short-term solution [12]. This approach requires that the two sensors are redundant in that they are not housed in the same probe. Otherwise, tissue compression or other local effects that can impact both sensors and signal disruptions would remain undetected. Use of duplicate sensors can identify occurrence of signal disruptions and erroneous glucose data.

Another proposed solution is to use sensors that are soft and small with microtextured surfaces in order to match tissue modulus [24]. This would both enhance biomechanical biocompatibility and help minimize reduce interfacial stress concentrations [24].

6 Conclusion

Despite several years of expert discussions, neither industry developers nor independent investigators have been able to standardize methodologies for evaluating CGM performance [37]. The one guideline that does exist only covers performance metrics, how studies should be designed and the data analyzed, but no acceptance criteria are provided [14]. Consequently, studies continue to use differing designs and metrics for assessing CGM accuracy, which makes comparisons of currently available CGM devices difficult.

Although use of CGM has been shown to provide significant clinical benefits compared to SMBG [6, 15, 20, 27, 38, 41, 48], sensor performance has slowed widespread adoption of this technology and still is a limiting factor in the development of closed-loop insulin delivery systems.

Because CGM system performance is currently evaluated against blood glucose measurements, these assessments do not reflect the true nature of CGM and may lead to misleading results. Nevertheless, because BG values are still regarded as the “gold standard,” accuracy must be assessed in comparison to BG values.

As discussed, the MARD is the most relevant accuracy metric for CGM, not only when computed for the overall glucose range but also separately for different glucose concentration ranges and different rates of change. However, because the MARD does not fully detect sensor performance due to the limited number of paired data points and/or the inherent physiological differences between the two body compartments sampled (i.e., blood glucose vs. interstitial glucose), calculation of the PARD becomes important as a measure for CGM precision and in supporting the accuracy assessment by the MARD.

The question “how good is good enough?” can only be answered as it relates to the intended use. When used as a standalone, adjunctive device for monitoring current and trending glucose levels, with no control over insulin pump infusion, current CGM devices are adequate for their intended use. However, when the CGM data are used for therapy decisions or the CGM device directly controls insulin delivery, accuracy, and precision become increasingly important depending upon the degree of control. For example, when CGM is linked to a low-glucose suspend (LGS) insulin pump, which automatically stops administering basal insulin when current or impending low glucose is detected, the risk of patient harm due to CGM device malfunction is relatively low. Conversely, when CGM is functioning as a component of a closed-loop insulin delivery system, patient risk is significantly increased because detection of an erroneous high-glucose reading would automatically prompt the insulin pump to administer unneeded insulin, resulting in hypoglycemia. This highlights the importance of effectively addressing the critical biomechanical factors (micromotion and pressure) and sensor-to-tissue interface to ensure accuracy, precision, and reliability in clinical use.

In summary, simple, straightforward and clinically significant standards and guidelines are needed to establish common rules for the development of safe and effective products using CGM technology. These rules should include standardized metrics for accuracy and precision, guidance for study designs and data analyses schemata, and acceptance criteria based on intended CGM use.