1 Introduction

There are several important considerations when assessing the usefulness of a novel cardiac output (CO) monitoring method: ‘Accuracy’ and ‘precision’ are used to describe measurement quality of a CO monitoring method statistically. A monitoring method can be accurate but not precise and vice versa. The true value of a monitored CO parameter generally remains unknown, since it usually cannot be directly determined, neither by a reference method, nor by the novel method. Nevertheless, in validation studies the novel method is usually compared with a reference method [1–7]. ‘Reproducibility’ and ‘trending ability’ are of interest [5–12]. Likewise reproducibility is an important factor for trending ability, yet a method might be reproducible but unable to track changes correctly. ‘Sources of error’, i.e., influencing factors related to individual patients impairing measurement by the tested CO monitoring method [8, 13–15], should be identified. And last, in order to justify usage of a monitoring method, a potential ‘clinical benefit’ of the measured parameters for the examined patient group should be assessed [16–20].

Optimal methods and optimal usage of these methods for continuous monitoring of CO are still evolving and remain controversial. For example, in adult medicine, a prognostic benefit by application of various continuous CO monitoring methods in high-risk patient groups was reported [16, 18, 20], whereas other studies found no benefit [17, 19]. Common practice in adult anesthesia is not uniform and relies on standard discontinuous cardiovascular monitoring in most cases [21]. Finally, it is logical to choose a stepwise approach depending on the disease severity of patients, ranging from conventional discontinuous via non-invasive continuous to invasive continuous CO monitoring methods [22].

In neonatal medicine, semi-invasive and invasive CO monitoring methods (such as trans-pulmonary thermo-dilution combined with pulse contour analysis) are not feasible because of the size of the required catheters [23]. Transthoracic echocardiography (TTE) may be used to assess CO in neonates non-invasively, but provides only punctual measurements, requires some operator training [24–26], and interferes with preferred minimal handling of the neonate. Theoretically, electrical velocimetry (EV) [27–31], a form of impedance cardiography, might be useful as a non-invasive and continuously applicable CO measurement method in this particular patient group. The EV method is based on measurement of heart-cycle related changes in transthoracic impedance to a transcutaneously applied high-frequency electrical current with low amperage applied along the thoracic aorta, which is used to calculate left ventricular stroke volume and thus provide automated continuous CO monitoring [27–29]. Most previous validation studies on EV (and other impedance cardiography devices) compared with different invasive and non-invasive CO monitoring methods have focused on equivalence between methods. These equivalence studies found conflicting data depending on the studied patient group, with better equivalence in more homogeneous or anesthetized patient cohorts [8, 10, 30, 32–35] and non-equivalence in inhomogeneous patient groups, with e.g., different cardiac anatomy [36–38] or inhomogeneous body fluid composition [39], and an influence of sex, height, increasing CO, and stroke volume [40]. A methodical issue in any CO monitoring method comparison involves the duration of sampling interval time, with better equivalence in validation studies using longer sampling intervals (i.e., more heart cycles) compared to studies using short sampling intervals [41]. The issue of influencing factors or sources of error using EV has been addressed in some adult [30, 40] and pediatric studies [8, 35, 36]. Further, some adult [42] and pediatric studies have assessed the trending ability of the EV method [8–12].

To address these issues of accuracy/precision, influencing factors, and trending ability, we previously reported a comparison of EV with TTE in various pediatric patient groups [8], and found an influence of mechanical ventilation, non-invasive continuous positive airway pressure ventilation, body weight, and secondary abdominal closure.

CO measurement in neonatal patients is problematic owing to varying physiological shunts such as PDA or PFO during the perinatal period. Thus, in neonates, left ventricular stroke volume (LVSV) multiplied by heart rate may not be equivalent to systemic blood flow because of these physiological shunts. This is a limitation of any invasive or non-invasive device for measuring CO via LVSV in neonates. In our previous study [8], we demonstrated a trend for an effect of an isolated PDA (p = 0.077) in determining LVSV when comparing EV with TTE, although only 5 % of the neonatal sample cohort had a PDA.

In the present study, we compared LVSV simultaneously determined by EV (LVSVEV) and TTE (LVSVTTE) and examined the effect of the presence of shunts (such as PDA or PFO) as influencing factors on accuracy of EV versus TTE in otherwise healthy non-ventilated neonates. We also addressed reproducibility based on repeated LVSV measurements in individual neonates.

2 Methods

2.1 Study design and setting

We performed a prospective, single-center (University Hospital Hamburg-Eppendorf, Germany; UKE), cross-sectional, observational study. Neonates were placed in the supine position under a radiant heater, with EV electrodes in position as described below. Two different observers operated EV and TTE, and verbally coordinated simultaneous recording of LVSV by EV and TTE to acquire three data sets of 3 × 3 identical consecutive heart beats. This procedure was repeated in triplicate in each neonate within a few minutes, resulting in 3 × 3 matching LVSVEV and LVSVTTE recordings on a single measurement day for each neonate.

2.2 Study population

Healthy preterm and term neonates who were born or treated at the UKE were eligible if they were breathing without respiratory support and were not on inotropes. Children with cardiac or aortic arch abnormalities (except for the presence of PDA or PFO) on TTE were excluded by a complete echocardiographic examination of all study subjects. PDA and/or PFO were documented if they were observed on TTE, otherwise they were regarded as absent. PDAs were insignificant according to TTE criteria [43]. All children were in a regular sinus rhythm. The study was approved by the local ethical board. Parental consent was obtained prior to data collection (from August 2010 to August 2011). This study was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments.

2.3 Electrical velocimetry (EV)

An Aesculon® monitor (CE 0123; Osypka Medical, Berlin, Germany) was used for LVSVEV determination. The positions of four RedDot® neonatal electrocardiogram (ECG) radiolucent pre-wired monitoring electrodes (3 M Health Care, Neuss, Germany) were chosen as recommended by the manufacturer on the left side of the neonate (forehead, lateral neck, mid-axillary line at the level of the xiphoid, and upper thigh). The signal generated by the Aesculon monitor for LVSV measurements by EV (LVSVEV) was accepted if visualization of the ECG and impedance curves on the Aesculon monitor showed acceptable signal quality and the green signal quality bar indicated a reliable signal. The device is described by US patent no. 6:511:438 B2, 28 January 2003. The algorithm is as follows:

$$ SV_{{{\text{B}}\text{-}{\text{O}}}} = \frac{{V_{\text{ITBV}} }}{{\zeta^{n} }}\sqrt {\frac{{{\text{d}}Z/{\text{d}}t_{\hbox{max} } }}{{Z_{0} }}} T_{\text{LVE(c)}} , $$

where VITBV = intrathoracic blood volume (mL), ζ = index of transthoracic aberrant electrical conduction, dZ/dtmax = peak rate of change of the blood resistivity (velocity) component of the transthoracic cardiogenic impedance pulse variation (ohmic mean acceleration) (Ωs−2), Z 0 = transthoracic base impedance (Ω), √((dZ/dtmax)/Z 0) = acceleration step-down transformation (s−1), and T LVE (c)= heart rate-corrected left ventricular ejection time (s) [29].

2.4 Echocardiography (TTE)

LVSVTTE was based on measurement of the velocity time integral (VTI) over the aortic valve (measured from an apical four-chamber view) multiplied by the area of the aortic valve. The aortic valve diameter was determined by triplicate measurements of the transaortic diameter, which was measured at the aortic valve (AOV) hinge points in the parasternal long axis view (AOV area = [0.5 × diameter]2 × 3.14). Therefore, LVSV = AOV area × VTI [24, 26]. For echocardiography, either a GE Medical Systems Vivid 7 (CE 0470) or a GE Healthcare Technologies Logiq P5 (CE 0459) (GE Healthcare, Munich, Germany) ultrasound machine was used. All TTE examinations were performed by a single operator.

2.5 Data collection and statistics

Data were analyzed using SPSS software (SPSS 20.0®; IBM, Armonk, NY, USA). Bland–Altman analysis comparing LVSVEV and LVSVTTE was performed on aggregated data points by combining measurements into a single averaged data point for each neonate [1, 2]. As described previously [3, 6, 7, 25, 34], bias was defined as the mean method difference of LVSVTTE and LVSVEV; limits of agreement (LOA) were defined as mean difference ±1.96 × SD of method difference; mean percentage error (MPE) was defined as MPE = (1.96 × SD of method difference)/((meanTTE + meanEV)/2); true precision for EV (TPEV) was based on an assumed true precision for TTE (TPTTE) of 30 % [25] and calculated as TPEV = √((MPE)2 − (TPTTE)2) = √((MPE)2 − (0.3)2). Subgroup analyses on samples with and without echocardiographic shunts (PDA and/or PFO) were performed. To make data comparable for different sizes of LVSVs, analysis of the effects of PDA and/or PFO was based on the relative difference between LVSVTTE and LVSVEV. Reproducibility over repeated measurements was determined using Kendall’s coefficient of concordance and by modified Bland–Altman analysis [1, 2] comparing the three different sets of three simultaneous consecutive heart beats in the individual patients. The coefficient of repeatability was calculated as 1.96 × SD of measurement difference between repeated measurements. An alpha error of 0.05 was considered statistically significant. A post hoc, two-sample, two-sided power calculation was performed comparing the mean percentage bias between LVSVTTE and LVSVEV in neonates with and without shunts.

3 Results

3.1 Sample cohort

Data from 291 simultaneous paired measurements of LVSVTTE and LVSVEV in 99 neonates (58 % male; median age 46 h, mean age 96 h, range 4–2160 h; see Table 1 for further details) were included. Each of the 291 paired LVSV measurements consisted of three simultaneous LVSV determinations over three consecutive heart beats. Three sets of the triplicate LVSV recordings (over three consecutive heart beats each) were obtained in 95 neonates, two triplicate LVSV recordings in two neonates, and one triplicate recording only in two neonates. In these 99 neonates, there was no visible shunt in 32/99 on TTE, 42/99 had a PFO alone, 4/99 had a PDA alone, and 21/99 had a PFO and a PDA in combination. Left to right shunts over PDA or PFO were insignificant for TTE criteria [43]. Hemodynamic characteristics of the sample cohort are summarized in Table 2. There were no adverse effects by EV. No patients developed skin lesions from the ECG electrodes used for the measurements.

Table 1 Patient and sample characteristics (99 neonates)
Full size table
Table 2 Hemodynamic characteristics of sample cohort (291 data points)
Full size table

3.2 Comparison of EV with TTE

By Bland–Altman analysis, the mean absolute LVSVEV was 5.5 mL compared with a LVSVTTE of 4.9 mL, resulting in an absolute bias of −0.7 mL (LOA −3.0 to 1.7 mL; relative bias −12.8 %; MPE 44.9 %; TPEV 33.4 %, n = 99 pooled data points). EV overestimated LVSV in relation to TTE, with a relative underestimation of LVSV for smaller LVSV values and overestimation of larger LVSVs (Fig. 1). Pearson’s correlation coefficient between TTE and EV in determining LVSV was significant (r = 0.68, p < 0.01, n = 99 pooled data points). The relative difference between LVSVTTE and LVSV EV was significantly correlated to length, weight, aortic valve diameter, LVSVEV, COEV, COTTE, CIEV, CITTE, no correlation was found to the gender and LVSVTTE (n = 291 data points).

Fig. 1
figure 1

Bland–Altman plot comparing measurement of left ventricular stroke volume (LVSV) by electrical velocimetry (EV) and transthoracic echocardiography (TTE). Absolute bias LVSVEV versus LVSVTTE −0.7 mL, limits of agreement −3.0 to 1.7 mL; relative bias −12.8 %; MPE 44.9 %; TPEV 33.4 %. Bias and 1.96 × SD are shown as reference lines. Pooled data on 291 paired measurements in 99 patients were included

Full size image

3.3 Effect of physiological shunts (PDA and/or PFO)

Analysis for the subgroup of neonates without visible shunts on TTE (n = 32) showed a mean absolute LVSVEV of 5.0 mL compared with LVSVTTE of 4.6 mL, resulting in an absolute Bland–Altman bias of −0.4 mL (LOA −2.8 to 2.0 mL; relative bias −8.2 %; MPE 50.7 %; TPEV 40.9 %), with a correlation coefficient of 0.76 (p < 0.01) between LVSVEV and LVSVTTE. The subgroup of neonates with shunts showed LVSVEV of 5.8 mL and LVSVTTE of 5.0 mL, resulting in an absolute bias of −0.8 mL (LOA −3.1 to 1.5 mL; relative bias −14.8 %, MPE 41.9 %, TPEV 29.3 %, n = 67), with a correlation of 0.60 (p < 0.01) between LVSVEV and LVSVTTE. The presence of a PDA and/or PFO led to a significant increase in the relative difference in LVSVEV versus LVSVTTE in paired measurements; with a relative difference of −2.9 % in subjects without shunts (n = 91), for subjects with a PFO alone −9.6 % (n = 125), PDA alone −14.0 % (n = 12), or PDA and PFO −18.5 % (n = 63) (see Table 3; Fig. 2). Further hemodynamic characteristics comparing neonates with and without shunts are summarized in Table 4. The correlation between the relative difference LVSVEV and LVSVTTE versus several hemodynamic and other neonatal parameters is summarized in Table 5.

Table 3 Influence of shunts on the relative difference in paired LVSV measurements
Full size table
Fig. 2
figure 2

Accuracy depending on presence or absence of shunts (PFO and/or PDA) comparing LVSVTTE and LVSVEV. The boxplot shows interquartile range, median, and outliers (291 paired measurements, for details see Table 3)

Full size image
Table 4 Hemodynamic characteristics for neonates without shunts (n = 91 paired samples) in comparison with neonates with shunts (PDA and/or PFO, n = 200 paired samples)
Full size table
Table 5 Pearson correlation and significances (two-tailed) between relative differences LVSVEV and LVSVTTE and hemodynamic and neonatal parameters, significant correlations in bold
Full size table

3.4 Reproducibility

Consecutive measurements of LVSV were performed for comparison of the EV and TTE methods. Kendall’s coefficient of concordance W was 0.986 (Chi square, 287.014; p < 0.001) for the percentage difference between LVSVEV and LVSVTTE over these three repeated measurements. This implies a constant direction of bias between the two CO monitoring methods. A graphic impression of reproducibility is given in Fig. 3. The coefficient of repeatability comparing the first and second sets of LVSVEV measurements with each other was 1.8 mL and comparing the second and the third LVSVEV measurements it was 1.9 mL. The corresponding coefficients of repeatability for LVSVTTE were 1.5 and 1.2 mL. Calculations according to Bland and Altman [2] based on the 291 triplicate paired measurements under the assumption of a constant true value did result in limits of agreement of −3.7 to +2.4 mL equivalent to a corresponding method related coefficient of repeatability of 1.1 mL for LVSVEV and 0.9 mL for LVSVTTE. If the equivalent calculations [2] were performed comparing only two LVSV measurement sets with each other, the corresponding limits of agreement were −3.6 to 2.0/−3.6 to 2.4 mL and the method related coefficients of repeatability were 0.9/1.0 mL for LVSVEV and 0.8/0.6 mL for LVSVTTE (first vs. second measurement set/second vs. third measurement set).

Fig. 3
figure 3

Bland–Altman plot showing reproducibility comparing two separate LVSVEV measurements. Mean difference LVSVEV1 versus LVSVEV2 and 1.96 × SD are shown as reference lines. The coefficient of repeatability was 1.8 mL (97 data points)

Full size image

3.5 Post-hoc power calculation

This study was sufficiently powered to detect the observed bias between LVSVEV and LVSVTTE between neonates with and without shunts, with a power (1 − β) of 91 % at a type 1 error rate (α) of 5 %. The power was 82 % with a type 1 error rate of 10 % to detect the observed bias between patients without shunts and those with an isolated PFO, and 96 % with a type 1 error rate of 5 % to detect the observed bias between neonates without shunts and neonates with a combination of PDA and PFO. For comparison between patients with no shunt and patients with an isolated PDA, the study was underpowered.

4 Discussion

This study compared LVSV measurements simultaneously recorded with EV and TTE methods in a neonatal, spontaneously breathing sample cohort. The main finding was a difference in accuracy based on the presence or absence of physiological left to right shunts (PDA and/or PFO) in the method comparison. The highest correlation and the lowest bias between the EV and TTE methods were observed in the subgroup of neonates without any shunt on TTE. Bias according to Bland–Altman analysis [1, 2] was below 10 % for neonates without shunts, but was higher than 10 % for neonates with shunts. The calculated MPE between LVSVEV and LVSVTTE was higher than 30 % for both groups. Therefore, due to the high MPE, the EV and TTE methods did not fulfil formal equivalence criteria [1–3] for any of the groups in our study setting.

The bias and MPE levels observed in the present study are consistent with a recent meta-analysis on EV and other non-invasive CO monitoring methods [44] showing an average MPE of 53.4 % for ultrasound blood flow measurements and 23.4 % for EV. The observed non-equivalence between EV and TTE in LVSV determination is also in concordance with our previous findings in the premature baby subgroup [8] based on a similar study design.

Our findings regarding the influence of physiological shunts are consistent with theoretical considerations on circulation and the EV monitoring method. Theoretically, the presence of PDA alone without PFO (or any other shunt) should increase LVSV and decrease right ventricular stroke volume, while the presence of PFO alone without PDA should have the opposite effect. The presence of both shunts should have a variable effect on LVSV and right ventricular stroke volume. The EV method depends on a variation of transthoracic electrical impedance during the cardiac cycle, with a decrease in trans-aortic impedance with laminar alignment of erythrocytes during systole [13, 15, 28, 29, 33]. Additional blood flow via an open PDA may theoretically result in an increase in LVSV as measured by EV, as the EV method cannot distinguish blood flow in the aorta from blood flow in a PDA (or any other vessel) flowing partially parallel to the aorta.

In the current study, the presence of a PDA (even though not hemodynamically relevant according to TTE criteria) resulted in a significant increase in bias between LVSVEV and LVSVTTE, with a relatively higher increase in LVSVEV than LVSVTTE. The higher LVSV values obtained by EV compared with TTE in the presence of PFO may be explained by the fact that EV measurements reflect changes in flow in any vessels parallel to the distance between the measuring electrodes, including systolic flow in the systemic and pulmonary arterial system [15]. Therefore, EV theoretically detects an increase in the combined LVSV plus right ventricular stroke volume output caused by any shunt. Our data support this assumption, as neonates with shunts had significantly higher LVSVEV, COEV, and CIEV measurements compared with those without shunts. Nevertheless, TTE data also showed increased LVSVTTE, COTTE, and CITTE measurements in neonates with shunts, albeit less marked. Interestingly the relative bias between LVSVEV and LVSVTTE was positively correlated to LVSVEV and the EV derived parameters COEV and CIEV, whereas the correlation was negative for the TTE derived parameters. This finding also supports the above theoretical considerations.

If the effect of PDA is analyzed without simultaneously considering PFO and vice versa, and this is not expressed as a percentage difference, an effect of these two types of shunts when comparing EV with TTE may be missed (this occurred in our initial data analysis presented in 2012) [45]. Compared with our study, in a considerably smaller cohort of only 20 children, Noori et al. [34] did not detect an effect of PDA on EV measurements. By contrast, in a recent study by Torigoe et al. [46] including 81 simultaneous CO measurements in 28 neonatal very low/low birth weight patients, a significant effect of hemodynamically relevant PDA was shown between EV and TTE. An effect of pathological shunts (ventricular septal defects) on EV was previously reported [36]. Thus, the present findings regarding the influence of left-to-right shunts on EV recordings are consistent with theoretical considerations and prior literature [8, 15, 36, 46, 47]. Further, our data suggest that shunts considered hemodynamically irrelevant according to echocardiography criteria via a PDA and/or PFO can affect accuracy of measurements of LVSVEV in comparison to LVSVTTE, thus identifying these shunts as a ‘source of error’ in EV recordings.

Reproducibility of EV was assessed comparing the three simultaneous triplicate LVSVEV and LVSVTTE measurement sets with each other. There was a high Kendall’s concordance with a constant direction of bias for the individual patients. The average bias between LVSVEV and LVSVTTE did not change with repeated measurements. The calculated coefficients of repeatability for EV were comparable to TTE. This implies that measurement method related scattering of individual measurements was comparable between EV and TTE. The analysis showed, that triplicate measurements (i.e., three sets of paired measurements) were not superior (regarding Bland–Altman limits of agreement) to duplicate measurements suggesting that two measurements instead of three might have been sufficient in this study setting. The short term reproducibility of LVSVEV measurements demonstrated in this study possibly implies long term trend following capability of the method EV [40, 41, 44], which was however not tested by the setting of this study.

4.1 Study limitations

There are several potential technical and statistical limitations in the present study. First, TTE is not a perfect reference technique [25]. There were potential TTE imprecisions as a result of Doppler probe position or angulation errors with potential underestimation of trans-aortic flow [24, 26], as TTE was based on a single plane only rather than two opposing planes (to achieve simultaneous paired LVSV measurements of identical heart beats by EV and TTE). However, for ethical and compliance reasons there is realistically no other method applicable in this study on healthy neonates. Second, there are potential EV imprecisions as some EV measurements in the observed cohort were inappropriately low, but were not excluded from the analysis because no clinical or echocardiographic cause was identified. A third possible technical limitation is that this study analyzed coupled LVSVEV versus LVSVTTE measurements, irrespective of the respiratory cycle. Our study did not take into account that stroke volume variation was between 10 and 30 % in most cases, and in vigorously breathing neonates it was up to 40 % in the reported cohort (as indicated by the Aesculon monitor). Stroke volume variation during the respiratory cycle is a problem for any method of determining CO [4]. Comparison of EV with TTE over a relatively short sampling interval of only three consecutive heart beats is vulnerable to measurement effects caused by the respiratory cycle [5]. Arrhythmias may also be more relevant with short sampling intervals [4, 28, 40, 41]. Nevertheless, the focus of our study was on simultaneous LVSV recordings by EV and TTE, which may help to compensate for the respiratory and heart rhythm effects, and the neonates were in sinus rhythm. A fourth potential limitation is that our study was not a formally blinded design with simultaneous recording of EV and TTE data. However, data acquisition by EV and TTE was performed by two separate operators who did not know the results of the other operator at the time of recording thus operating independently and practically blinded. A final potential limitation is the uneven distribution of the shunts over the sample cohort. However, a post hoc power analysis showed sufficient power to compare the percentage bias in neonates without any shunts with the group of neonates with shunts.

5 Conclusion

The non-invasive and continuously applicable EV method is feasible for monitoring LVSV in neonates, but is not formally equivalent to TTE. This study shows that physiological left to right shunts via PFO and/or PDA in neonates are a source or error affecting accuracy of LVSVEV measurements in comparison with LVSVTTE measurements, resulting in an increased bias and an overestimation of LVSV by EV.