1 Introduction

The microcirculation is altered in several pathological conditions and the direct visualisation of the microvascular network is becoming of increasing interest for clinicians, particularly in the context of critically ill patients. Orthogonal polarization spectral (OPS) and side stream darkfield (SDF) are two video-microscopic imaging techniques that can be used at the bedside to visualise the microcirculation. Dedicated software is available to calculate capillary density and flow parameters. However, analysis of microvascular parameters is offline, time consuming and operator dependent [1, 2] as it requires the intervention of the operator to correctly identify each capillary and to score blood flow. This represents the main limitation of using microvascular imaging in clinical practice as a “point-of-care” tool. Only microcirculatory flow index (MFI) has been evaluated in real-time [3]. Recently a third generation handheld microscope based on incident dark field (IDF) imaging has been introduced (CytoCam, Braedius Medical, Huizen, The Netherlands). This technique is able to provide higher quality images than SDF imaging, visualizing approximately 20–30 % more capillaries than SDF device [4–6]. This camera is provided with an automatic analysis software (CytoCamTools 1.7.12, Braedius, Huizen, The Netherlands) that could allow us to obtain results quickly. However, the ability of this new software to obtain microvascular parameters in comparison to classical manual analysis performed with other analysis system such as Automated Vascular Analysis software 3.2 (MicroVision Medical, Amsterdam, The Netherlands) is unknown. In 2007, a consensus conference on the evaluation of the microcirculation proposed two density variables and two perfusion indices for the analysis of microcirculation [1]. The density variables are the total vessel density (TVD) and the perfused vessel density (PVD). PVD for small vessels (diameter <20 micron) provide an estimate of functional capillary density, the main critical factor of tissue perfusion. The perfusion indices are the proportion of perfused vessels (PPV) and MFI. PPV provides information about flow heterogeneity within the image and MFI discriminates between different type of flow (continuous, sluggish, intermittent, absent).

The objective of this study is to assess the ability of the new automatic analysis software CytoCamTools 1.7.12 (CC) to obtain the microvascular parameters proposed by the consensus conference and its efficiency in comparison with Automated Vascular Analysis (AVA) software 3.2.

2 Materials and methods

This study was approved by the Camden and Islington ethic committee (13/LO/1307, December 2013) and by St. George’s University of London. Written informed consent was obtained from all patients.

2.1 Participants

Patients admitted to the cardiothoracic intensive care unit (ICU) following cardiac surgery were prospectively enrolled. Patients with occlusive peripheral vascular disease, post-operative valvular regurgitation, presence of an intra-aortic balloon pump, pregnancy, body weight below 50 kg, absence of radial arterial catheter, evidence or strong suspicion of active bleeding or sepsis and patients requiring aggressive fluid resuscitation or changes in vasoactive therapy were excluded from the study. All patients were studied during the initial period in the ICU, once they achieved hemodynamic and respiratory stability, defined by changes no >10 % in heart rate, arterial pressure and arterial saturation of oxygen during 10 min.

2.2 Measurements and study design

Sublingual microcirculatory videos were obtained using an IDF-imaging device (CytoCam, Braedius medical, Huizen, The Netherlands). The camera was gently applied without pressure after gentle removal of secretions with gauze. The microcirculation image quality score (MIQS) was used to assess the images quality [7]. Briefly, a score of optimal (0), suboptimal but acceptable (1), or unacceptable (10) was assigned to six categories (illumination, duration, focus, content, stability, and pressure). Any video with a cumulative score of 10 or higher is classically considered unacceptable for further analysis. Since Massey et al. [7] reported a mean MIQS of 1.68 (±0.90) for videos passing the quality analysis, only videos with a MIQS of 2 or less were considered eligible for the comparison between the two softwares. Each video was blindly analysed using AVA 3.2 by two skilled operators (with an experience of more than 700 videos analysed each) and results were compared with the analysis obtained by the CytoCamTools 1.7.12 (CC) software. For each video, AVA and CC software calculated TVD (mm/mm2) for small vessels (diameter <20 micron), PVD (mm/mm2) and PPV (%), according to the current international consensus [1]. Briefly, TVD was assessed with AVA drawing manually each vessel. Perfusion was assessed by the operator: vessels with continuous or sluggish flow were considered perfused and vessels with intermittent (at least 50 % of time with no flow) or absent flow were considered not perfused. PPV was calculated as the ratio between perfused vessels and TVD. PVD has been calculated by multiplying TVD by PPV. MFI was calculated using AVA software dividing the image in four quadrants and determining the predominant type of flow in each quadrant assigning a score between 0 and 3 (0: absent; 1: intermittent; 2: sluggish; 3: normal). The MFI value for the whole image derived from the average value of the single quadrants (Fig. 1). CC software assesses flow using the averaged perfused speed indicator (APSI). This is a quantitative estimation of red blood cells velocity, based on calculation of a dimensional index (speed indicator or SI) with values between 0 and infinite. This is derived from the intensity variation in the pixels along the central line of the vessel. Vessels with an SI higher than 1 are considered perfused and for these vessels the software calculate the APSI (Fig. 2). The SI threshold value of 1 is used by CC to calculate PVD and PPV, as it discriminates perfused from not perfused vessels.

Fig. 1
figure 1

Microvascular flow index (MFI) was calculated dividing the image into four quadrants and determining the predominant type of flow in each quadrant assigning a score between 0 and 3 (0 absent; 1 intermittent; 2 sluggish; 3 normal). The MFI value for the whole image derived from the average value of the single quadrants

Full size image
Fig. 2
figure 2

Calculation of averaged perfused speed indicator (APSI). a CC software assign a Speed indicator for each vessel (SI). b Then, those vessels with a SI >1 are considered perfused. c Only those vessels with a SI >1 are used to calculate the APSI

Full size image

As a quality measure, 20 videos were analysed independently by both operators to detect inter-observer variability.

Duration of analysis was recorded for 40 videos (20 for each system), from the selection of the video until the visualisation of numerical results.

2.3 Statistical analysis

Descriptive statistics and graphics were used to understand the data structure, the nature of the variables and the appropriateness of the subsequent tests and analysis. Continuous variables are summarized by their means, medians, standard deviations (SD) and/or inter-quartile range (IQR). Bland–Altman analysis adjusted for repeated measures [8, 9] was implemented using a mixed model for repeated measures [10], where the fixed factor is the type of software adjusted by the mean of all the observations per subject as covariate. The estimate coefficient for the type of software provides the mean bias between CC and AVA software, with the correspondent 95 % confidence interval. The assumption that the mean of the repeated measurements is independent of the difference of two measurements and the variance of the repeated measurements was checked.

The percentage error (PE) was calculated from the ratio of 1.96 times standard deviation of the mean bias between the mean value of the variable of interest (i.e. TVD) [11]. Since previous studies have shown differences in microvascular parameters between healthy volunteers and septic patients ranging from about 20–60 % [12, 13], we considered a PE value <30 % as acceptable.

Pearson correlation coefficients for TVD, PVD and PPV values obtained with the different softwares are reported. Given that repeated measurements for each study subject were obtained, an ordinary correlation coefficient is not appropriate [14]. For these variables, a “within subjects” correlation coefficient was reported, which accounts for the lack of independence among the repeated measurements by removing the variation between subjects [15]. A within subjects correlation coefficient examines whether an increase in a variable (i.e. TVD with CC) within the same individual is associated with an increase in the other variable (i.e. TVD with AVA) [15]. In addition, a “between subjects” correlation coefficient is reported, where the repeated measurements were first converted to means, so that each subject contributed only with one observation. The Pearson correlation coefficient was then calculated as a weighted average correlation coefficient, weighted by the number of repeated measurements originally available for each subject. The p values were computed, however, based on the number of subjects, or means, rather than the original number of repeated measurements. This “between subjects” correlation coefficient examines whether subjects with a high value on one variable also tend to have a high value on the other variable, similar to the ordinary correlation coefficient based on independent observations [16].

In addition to the previous analysis, the coefficient of variability was calculated to assess the inter-observer variability. The Wilcoxon test (z) was used to compare paired means of non-normally distributed variables. A mixed model was used to compare the mean APSI with the observed MFI category.

Independent-samples Mann–Whitney U test was used to compare analysis duration time between the two software. Times are reported in minutes and seconds (mm:ss). A p value <0.05 was considered for statistical significance. MedCalc 12.3 (MedCalc Software, Belgium) and SPSS Statistical Software version 22 (SPSS, IBM, USA) was used for the statistical analysis.

3 Results

440 videos from 25 patients were recorded. 356 videos were excluded because they did not meet the highest quality in terms of illumination, focus, content, brightness and stability. Therefore, 84 videos from 22 patients were analysed (Fig. 3).

Fig. 3
figure 3

Flow diagram showing videos selection

Full size image

3.1 Total vessel density

The mean value of TVD was 21.56 ± 3.70 mm/mm2 for AVA and 19.39 ± 3.29 mm/mm2 for CC. The mean bias between TVD-CC and TVD-AVA was 2.20 mm/mm2 (95 % CI 1.37–3.03; p < 0.001) with limits of agreement (LOA) of −4.39 (95 % CI −5.66 to −3.16) and 8.79 (95 % CI 7.50–10.01) mm/mm2 (Fig. 4). The PE for TVD was ±32.2 %.

Fig. 4
figure 4

Bland–Altman plot of the difference between TVD measured with Automated Vascular Analysis (TVD AVA) and TVD measured with CytoCamTools (TVD CC) against the mean of TVD AVA and TVD CC in the 22 patients in the study. The within-subject variance is estimated by a random effects model, which includes the mean measurements of the two methods for each measurement occasion. SD standard deviation

Full size image

TVD was positively correlated between CC and AVA (r = 0.39, p = 0.001) in a “within subjects” correlation and for the “between subjects” correlation (r = 0.74, p < 0.001; Fig. 5).

Fig. 5
figure 5

TVD weighted correlation between AVA and CytoCamTools (r = 0.74, p < 0.001)

Full size image

3.2 Perfused vessel density

The mean value of PVD was 20.91 ± 3.79 mm/mm2 for AVA and 14.36 ± 4.16 mm/mm2 for CC. The mean bias between PVD-CC and PVD-AVA was 6.54 mm/mm2 (95 % CI 5.60–7.48; p < 0.001) with LOA of −4.25 (95 % CI −8.48 to −0.02) and 17.34 (95 % CI 13.11–21.57) mm/mm2 (Fig. 6). The PE for PVD was ±61.2 %.

Fig. 6
figure 6

Bland–Altman plot of the difference between perfused vessel density (PVD) measured with Automated Vascular Analysis (TVD AVA) and PVD measured with CytoCamTools (TVD CC) against the mean of PVD AVA and PVD CC in the 22 patients in the study. The within-subject variance is estimated by a random effects model, which includes the mean measurements of the two methods for each measurement occasion. SD standard deviation

Full size image

PVD was positively correlated between CC and AVA (r = 0.35, p = 0.005) in a “within subjects” correlation as well as for the “between subjects” correlation (r = 0.66, p < 0.001; Fig. 7).

Fig. 7
figure 7

PVD weighted correlation between AVA and CytoCamTools (r = 0.66, p < 0.001)

Full size image

3.3 Proportion of perfused vessels

The median PPV obtained with AVA was higher than the median PPV obtained with CC [97.39 % (IQR: 95.25, 100 %) vs. 81.65 % (IQR: 61.97, 88.99 %), z = 7.92, p < 0.0001].

Bland–Altman plot was not drawn for PPV because the assumptions to apply this statistical method were not met (the differences are proportional to the mean [8]). The Fig. 8 shows the distribution of the PPV obtained with AVA and with CC software.

Fig. 8
figure 8

Box-and-Whisker plot for PPV (p < 0.0001). PPV proportion of perfused vessels, AVA Automated Vascular Analysis, CC CytoCamTools 1.7.12

Full size image

PPV was neither correlated between CC and AVA in a “within subjects” correlation (r = 0.12, p = 0.35) nor in “between subjects” correlation (r = −0.05, p = 0.65).

3.4 MFI and APSI

The median value of MFI was 3 (IQR: 3, 3) and the median value of APSI was 6.49 (IQR: 5.46, 8.03).

The Fig. 9 plots the value of APSI and MFI for each quadrant. There is no evidence that MFI categories can estimate or predict APSI values (F = 0.93, df = 4/74.2, p = 0.45).

Fig. 9
figure 9

APSI of quadrants by different values of MFI. MFI microvascular flow index

Full size image

3.5 Inter-observer agreement

Twenty videos were analysed independently and blindly by two operators using AVA software. The Table 1 shows the inter-observer agreement between the two operators.

Table 1 Inter-observer analysis
Full size table

3.6 Time analysis

The time required for the analysis was shorter with CC than with AVA system [2:42 (IQR: 2:12, 3:31) vs. 16:12 (IQR: 13:38, 17:57), U = 0.0, p < 0.001; Fig. 10].

Fig. 10
figure 10

Box-and-Whisker plot for time of analysis (p < 0.001)

Full size image

4 Discussion

There are two basic variables in the analysis of microvascular circulation images: detection of vessels (TVD) and quantification of blood flow (MFI or APSI). This study shows a significant bias between CC and AVA to assess TVD and a fairly good correlation between the two systems for this parameter. However, this is close to the current agreement between two operators using the same system. Regarding flow quantification, no relationship between APSI values and different MFI categories was found. For the other two mixed variables (PVD and PPV) significantly greater bias was observed for PVD and no correlation was found between the two systems for PPV. Finally, the time required for the analysis was shorter with CC than with AVA system.

Few studies have been conducted until now using the IDF-imaging technology. Aykut et al. [4] acquired sublingual microvascular videos using IDF and SDF cameras in 25 healthy volunteers to compare the parameters and the quality of the images obtained by the two devices. AVA software has been used for the analysis of all images. IDF camera was able to show a higher vessel density than SDF (TVD 21.60 ± 4.30 mm/mm2 vs. 16.35 ± 2.78 mm/mm2, p < 0.0001; PVD 21.50 ± 4.38 mm/mm2 vs. 16.24 ± 2.81 mm/mm2, p < 0.0001). These results were due to the best contrast, sharpness and overall quality of image when IDF device was used. Van Elteren and co-workers [5] also showed higher values for TVD and PVD when IDF camera was used to analyse subcutaneous microcirculation in 20 neonates (TVD 14.8 ± 2.1 n/mm vs. 12.5 ± 1.8 n/mm, p = 0.001; PVD 13.0 ± 2.1 n/mm vs. 11.6 ± 1.9 n/mm, p = 0.033). These studies demonstrated the improvement in image quality using IDF camera, however the automatic analysis software CC has not been assessed against the current available technology. To our knowledge, this is the first study that compared the automatic software CC and AVA.

The values of TVD are fairly similar between the two systems and showed a marginal bias. The LOA were slightly higher than the one between the two independent observers. In practice, these two values (TVD-CC and TVD-AVA) are interchangeable with the advantage of having a faster acquisition for CC compared to AVA. This essentially means that the automatic software is as accurate as the investigator in detection of vessels, but much faster. The moderate “within subject” correlation found for TVD could partially compensate the lack of agreement between the two systems as the ability to follow a trend in case of repeated measurements might be of greater clinical interest than the agreement of absolute values.

The values of PVD are different between the two systems and the good “between subject” correlation between the systems should not make us forget a bias of 6.54 mm/mm2. The “within subject” correlation, although significant, is just moderate. Actually, there is little agreement between CC and AVA for all the flow related variables (PVD and PPV). The main explanation for this could be the different methodology used for flow assessment. AVA uses a categorical variable (MFI) assigned by the operator according to the visual subjective judgement of flow vessel by vessel whereas CC uses a quantitative estimation of red blood cells velocity (SI) based on the intensity variation along the central line of the vessel. MFI takes into account all vessels (perfused and not perfused) to categorise the prevalent type of flow. APSI is calculated only for perfused vessels (the average value of SI higher than 1) and it does not include the SI values of not perfused vessels. This difference might explain why the two parameters are not related at all. Although PSI does not measure speed expressed in m/s, it is meant to objectively quantify flow in perfused vessels. Two considerations must be mentioned before jumping to strong conclusions: (1) There is no external validation of the new software and its underlying physical principles are not yet published in the literature, therefore the accuracy of the CC tools algorithm to measure flow is unknown and (2) there is no “gold-standard” technique to measure microcirculatory flow, therefore the superiority of one or other technology remains unclear. These two points makes data interpretation very difficult. CC software present us the same microvascular parameters obtained with AVA, which make the user assume the results can be interpreted in the same way, although the methods for flow estimation are different. Our study point out important differences in the interpretation of the parameters of these two systems.

In addition, another possible explanation is that the area analysed by the two software for each videos is not the same. In fact, CC is able to analyse a bigger sublingual area than AVA (1.55 × 1.16 mm vs. 0.94 × 0.75 mm respectively). Even if the density parameters are indexed for surface area, this difference should be considered, especially in presence of high density and flow heterogeneity within the same video.

The accuracy of CC analysis is greatly related to the image quality. Very high quality is needed to obtain acceptable result even for TVD, for which flow detection is not required. Ideal conditions are not always possible to be meet in daily clinical practice. Focus, contrast, brightness and especially image stability are crucial conditions for good results. Massey et al. [7] performed quality assessment of 59 videos and showed that about 46 % of them were acceptable for analysis (MIQS < 10). Since we do not know whether the cut-off of 10 for MIQS can be acceptable also to obtain appropriate results using automatic software, for the purpose of our study we decided to choose videos with highest quality (MIQS < 2) to limit the error due to artefacts. For this reason, we finally selected about 20 % of recorded videos. Thus, our results come from a highly selected sample of videos and we cannot exclude to find different results if less strict criteria will be used for video selection. Nevertheless, image quality is crucial to obtain reliable results also with AVA and we would expect that the performance of any software would not be better under suboptimal quality conditions.

Last but not least, we must consider the time needed for analysing the images. About 20 min are needed for off-line analysis of each video using AVA because the operator has to draw the vessels and classify the flow. This is the main reason that prevents microvascular assessment at bedside in routine clinical practice as “point of care” tool. On the contrary, only few min are needed to CC to analyse one video (about 3 min), the process is completely automated and does not require the intervention of the operator. According to the international consensus [1], at least three videos from three different sublingual sites should be analysed at each time point. This means that the overall analysis of a single patient using AVA will require about 1 h. On the other hand, CC will be able to give us results in less than 15 min. This is an important aspect, in order to introduce microcirculation imaging in clinical practice.

The variability between the operators was in line to the previous literature [12, 17] and we can use these data as benchmark for PE interpretation. Our study showed that the software error for TVD was performing at acceptable levels, compared to the results obtained by inter-observer agreement, but was still distant for PVD.

Should the new software replace the previous one? Given the differences in flow assessment, our study cannot answer the question but the progress is definitively promising. The results are not interchangeable for flow related parameters, and further research should focus on a system to assess red blood cells velocity objectively or new parameters independently from human-eye assessment.

Our study has some limitations. First, according to AVA software, we had only videos with good perfusion and we were not able to perform the comparison when an impaired perfusion is detected. Then, we assessed the agreement between the two softwares only regarding density parameters and we have not compared the ability to detect flow calculating the velocity of red blood cells. This analysis might give us a full explanation of the results shown in our study. Second, only videos with very good quality were included in the analysis, which can reduce the external validity of our results. Third, we performed inter-observer analysis only for AVA software. However, it is also interesting to know if the new software gives us the same results analysing the same videos several time. Finally, due to the great variability of the measures, the potential for a beta error is real.

5 Conclusions

In conclusion, the determination of TVD is comparable between the two software systems, although faster with CC software. The values for PVD and PPV are not interchangeable given the different approach to assess microcirculatory flow. TVD can already be monitored automatically at the bedside. Further developments are required to see if PVD and PPV could become point of care testing.