Keywords

1 Introduction

Retinal Image Analysis (RIA) aims to develop computational and mathematical techniques for helping clinicians with the diagnosis of diseases such as diabetes, glaucoma and cardiovascular conditions, that may cause changes in retinal blood vessel patterns like tortuosity, bifurcations, variation of vessel width and colour [1, 18]. RIA algorithms have to be validated to avoid obtaining misleading results. Validation can be defined as the process of showing that an algorithm performs correctly by comparing its output with a reference standard [16]. A common practice for validation of medical image algorithms is to use Ground Truth (GT) provided by medical experts. Obtaining manually GT images annotated by clinicians is an expensive and laborious task which motivates the creation of synthetic datasets providing GT of adequate quality for algorithm validation. Medical phantoms are used extensively in medical imaging [3, 9]. However, to our best knowledge, there are no publicly available databases of synthetic retinal fundus images, and providing annotations for large image repositories remains impractical (e.g. UK Biobank alone stores fundus images for 68,000 patients). Synthesized high-resolution fundus images, along with GT free from inter-/intra-observer variability, would allow an efficient validation of algorithms for segmentation and analysis of retinal anatomical structures: by tuning morphological and textural characteristics of these images, we can represent the hallmarks of several diseases or different populations. This work focuses on the generation of retinal vessels and their integration with non-vessel regions, i.e. retinal background, fovea and Optic Disc (OD), previously reported by Fiorini et al. [5], to yield complete fundus camera images. The resulting synthetic retinal fundus images include explicit GT for vessels binary maps, bifurcation point locations, vessel widths and artery/vein classification.

This paper is organized as follows. In Sect. 2 we describe the proposed method for the generation of the morphological properties (Subsect. 2.2) and the textural features (Subsect. 2.3) of the vasculature. In Sect. 3 we report results and summarize and discuss our experiments to evaluate them. Finally in Sect. 4 we give concluding remarks and hints for future work.

2 Method

2.1 Overview

The proposed approach consists of a learning phase and a generation phase. In the former phase, data describing vascular morphology and texture are collected from annotations of real images. Models are specified and their parameters learned from the training data. In the latter phase, the models obtained are used to create synthetic vascular networks. Arteries (A) and Veins (V) are created separately with the same protocol, and then combined together. This work is based on the publicly available High-Resolution FundusFootnote 1 (HRF) images database [13], and on a subset of retinal images of the GoDARTS bioresourceFootnote 2.

2.2 Vascular Morphology

The generation of synthetic vessel morphology has been achieved using the well-known Active Shape Model (ASM) [4]. This model provides a statistical representation of shape represented by a set of points, called landmark points. By analysing the variations in shape over the training set, a PCA model is built. The training samples (vessel centerlines in our case) are aligned into a common coordinate frame and the deviations from the mean shape are analysed. Each training shape is represented as a fixed number n of landmark points placed along a vessel and equally spaced. These landmarks form a 2n vector \(\mathbf {x}\), the dimensionality of which is reduced using PCA, assuming that the most interesting feature is the one with the largest variance. Hence, each shape can be approximated as:

$$\begin{aligned} \mathbf {x_i} \approx \mathbf {\bar{x}}+\mathbf {P}\mathbf {b_i} \end{aligned}$$
(1)

where \(\mathbf {\bar{x}}\) is the mean shape of the aligned data, \(\mathbf {P}\) contains the first t eigenvectors corresponding to the largest t eigenvalues of the covariance matrix of the training shapes, and \(\mathbf {b_i}\) is a t dimensional vector of parameters of a deformable shape model. We choose t, so that the model represents 98 % of the total variance of our training data. By varying the element in \(\mathbf {b_i}\), randomly choosing them from a multivatiate normal distribution learned across the training set shapes, we generate a new synthetic vessel using Eq. (1).

The data describing the shape of the vessels of each type (A and V) for the main arcades, nasal and temporal (\(n=81\) landmarks), and their branches (\(n=31\) landmarks) up to three levels of branching have been previously collected from 50 GoDARTS retinal fundus images. We used polar coordinates centred in the OD and with the main axis in the direction of the OD-Fovea axis (i.e. the line connecting the OD centre and the fovea), adopted by the VAMPIRE software suite [15]. Vessel shapes are represented into this system using a transformation that includes a rigid translation and rotation. Similarly the shapes of the branches have been aligned using a rigid transformation that shifts their starting point to the origin of the same coordinate system. Fig. 1 shows the aligned set of shapes and the mean shape for the temporal arcade (a) and a vessel branch (b).

Fig. 1.
figure 1

Vessels aligned shapes (green) and their mean shape (red) for the temporal arcades (a) a vessel branch (b). (Color figure online)

Full size image

Individually generated synthetic vessels are then connected to create the vascular network skeleton. The location of vessel bifurcations is estimated from real images as follows. First we calculate the spatial density distribution map (Fig. 2(a)) of all bifurcation points annotated on real images. Then we map our synthetic vessel onto it, obtaining a probability score for each point of the vessel to become a bifurcation point (Fig. 2(b)). We select one of the points having maximum score as the first bifurcation point of the main arcades. We select the following bifurcation point as one of the points having maximum score located at a distance \(d \in [ l/2n,l/n]\) from the previous one, where l is the length of the vessel and n is the desired number of bifurcations. These bounds are required to obtain a biologically plausible structure. We continue to select points until the desired number of bifurcation points is achieved. This number is chosen empirically, based on accurate analysis of the number of bifurcations in real images.

Fig. 2.
figure 2

(a) Density Map distribution of artery bifurcation points in the image plane. (b) A synthetic vessel with the probability score of each point to be a bifurcation point.

Full size image

We compute orientation and calibre for each branch vessel originating at a bifurcation point using Murray’s bifurcation model  [12], linking branching angles with vessel calibres. Newly generated synthetic branches need to fit with the context of the vascular tree already generated: all vessels should be inside the Field of View (FOV), but outside the foveal region, avoiding intersections between vessels of the same type, and converging toward the fovea.

The binary map of the vascular tree (see example in Fig. 3) is obtained by adding calibre information using mathematical morphological dilation of the skeleton. The initial calibre of the main arcades is sampled from the estimate distribution of the largest vessel calibre of real images. The initial calibre of the branches is obtained from the parent vessel calibre following Murray’s Law.

Fig. 3.
figure 3

Example of synthetic vascular tree (arteries in red and veins in blue for display purpose). (Color figure online)

Full size image

2.3 Vascular Texture

To generate synthetic vessel textures we collected examples of intensity values along vessels and of textural features of the surrounding area (background). We then created a model combining both information, capturing the transition of intensities between vessels and background.

Data Collection. Cross-sections of the vessel of interest were defined, spaced by 5 pixels along the vessel centerline. We extracted the intensity RGB profile on lines perpendicular to the direction of the vessel as depicted in Fig. 4(a).

Fig. 4.
figure 4

(a) Cross-sections perpendicular to vessel direction, background regions and RGB intensity profile along one of the cross-sections. (b) Green channel fitting profile using the Extended Multiresolution Hermite Model. (Color figure online)

Full size image

The green channel intensities are fitted (Fig. 4(b)) with a weighted NonLinear Least Squares model using a 6-parameters Extended Multiresolution Hermite Model [11] (EMHM) to fit the cross-sectional intensity profiles. The EMHM accounts for non-symmetric and symmetric profiles, with or without central reflex, expressed by the formula:

$$\begin{aligned} \begin{array}{lcl} H(a,m,\delta ,\sigma ,p,q,x)=-p\{1+a[(x-m-\delta )^2-1]\}e^{-\frac{(x-m)^2}{2\sigma ^2}}+q \end{array} \end{aligned}$$
(2)

where \(a \in [-1,1]\) models the depth of the central reflection; \(m \in [1, \textit{length(profile)}]\) is the mean of the Gaussian and allows shifts along the x-axis and length(profile) is the length of the vessel region around the target location; \(\delta \in [-2,2]\) accounts for asymmetry; \(\sigma \in [1,15]\) is the standard deviation of the Gaussian; \(q \in [0,255]\) shifts the function along y-axis, avoiding negative pixels values; \(p \in [0,150]\) guarantees that vessels are darker than the background; x is a vector of the same length of the cross-section of the vessel. The initial conditions are \(a=0\), \(m=\textit{length(profile)}/2\), \(\delta =0.2\), \(\sigma =\textit{length(profile)}/std(\textit{profile})\), \(q=max(\textit{profile})\), \(p=max(\textit{profile})-min(\textit{profile})\).

At the endpoints of each cross-section (green circles in Fig. 4(a)) we computed five statistical texture descriptors [7, 14] (Mean, Std, Skewness, Kurtosis and Entropy) on two near-circular windows of 6 pixel radii.

The ensemble of these data, 6 EMHM parameters (\(\mathbf {X}_{n \times 6}\)) and \(5 \times 2\) background texture descriptors (\(\mathbf {Y}_{n \times 10}\)) for each profile, for a total of 975 artery and 1593 vein profiles, collected from the 15 healthy subjects of HRF dataset, constitute the measurements for the procedure proposed below.

Generation of Vessel Textures. The procedure for creating reliable synthetic vessel texture takes into account both the continuity of intensity profiles along the vessel and their consistence with background intensities. We apply a Kalman Filter [8], casting our problem as a state space system:

$$\begin{aligned} {\left\{ \begin{array}{ll} \mathbf {x}_k=\mathbf {F} \mathbf {x}_{k-1}+\mathbf {w}_{k-1} &{} \text {System model} \\ \mathbf {y}_k=\mathbf {H} \mathbf {x}_k+\mathbf {v}_k &{} \text {Measurement model} \\ \end{array}\right. } \end{aligned}$$
(3)

where \(\mathbf {x}_k\) is the state vector containing the 6 parameters describing the intensity profile, \(\mathbf {F}\) is the state transition matrix (set to identity matrix), \(\mathbf {y}_k\) is the vector of measurements given by 10 textural descriptors of the synthetic background and the two vectors \(\mathbf {w}_{k-1}\) and \(\mathbf {v}_k\) are unrelated realizations of white zero-mean Gaussian noise. The measurement matrix \(\mathbf {H}\) has been obtained, using Multivariate Multiple Linear Regression, solving the system:

$$\begin{aligned} \begin{bmatrix} y_{1,1}&\dots&y_{1,10} \\ \vdots&\ddots&\vdots \\ y_{n,1}&\dots&y_{n,10} \end{bmatrix} = \begin{bmatrix} x_{1,1}&\dots&x_{1,6} \\ \vdots&\ddots&\vdots \\ x_{n,1}&\dots&x_{n,6} \end{bmatrix} \begin{bmatrix} h_{1,1}&\dots&h_{1,10} \\ \vdots&\ddots&\vdots \\ h_{6,1}&\dots&h_{6,10} \end{bmatrix} +\begin{bmatrix} \epsilon _{1,1}&\dots&\epsilon _{1,10} \\ \vdots&\ddots&\vdots \\ \epsilon _{n,1}&\dots&\epsilon _{n,10} \end{bmatrix} \end{aligned}$$
(4)

where the matrices \(\mathbf {X}_{n \times 6}\) and \(\mathbf {Y}_{n \times 10}\) are the measurements calculated as described in Sect. 2.3 and \(\mathbf {\epsilon }\) represents the system error.

Equation (3) recursively estimate, through a predictor-corrector method, the state \(\mathbf {x}_k\) and its covariance \(\mathbf {P}_k\). The initial estimate of the state \(\hat{\mathbf {x}}_0\) (first profile) is assumed to be known and its covariance matrix \(\mathbf {P}_0\) is initialized to zero. The first profile for the major arcades is the profile having background descriptors more similar to the current synthetic ones. The first profile for the branches is the profile of the parent vessel at the bifurcation point from which they originate.

Iterating this procedure, each new intensity profile of the green channel is generated taking into account the previous one and the surrounding background. A similar procedure was applied to the red and blue channels. However, based on experimental results, we later decided to simply use the average intensity profile of the training ones for the latter two channels. The red component is weighted with underlying background red intensity level, in order to take into account the spatial colour distribution of the whole image. Finally, the RGB intensity profile is cut with the full-width-at-half-maximum algorithm [10] to keep the mere component of the vessel, and re-sampled using the Bresenham line-drawing algorithm [2]. Experiments showed that the quality of the synthetic images generated would not improve using the full Kalman estimator in the red and blue channels.

The two vascular trees obtained (arteries and veins) are combined and superimposed on synthetic backgrounds, reported elsewhere [5], to create complete synthetic fundus camera images. Gaussian filtering is finally applied to smooth vessel edges and in general unrealistically sharp intensity changes.

The image size of the final synthetic is 3,125 \(\times \) 2,336 pixels with FOV diameter of 2,662 pixels, in line with the resolution of state-of-the-art fundus cameras. The whole method and an user-friendly GUI of the simulation tool have been implemented in Matlab\(^{\textregistered }\)2014b. An extended dataset of synthetic images and the simulation tool will be publicly available after publication.

3 Results

In Fig. 5 we visually compare real images (a, b) with synthetic images (c, d). We notice that the synthetic vessels are characterized by a realistic morphology, including typical tortuosity. The temporal segments of the arcades go toward and around the macula, and the nasal segments radiate radially from the nerve head. The vessels colouring is always darker than the background, following real images: vessels appear brighter around the OD and darker towards the fovea and the extremities of the FOV. The arteries appear, as in real images, brighter and narrower than veins. Because of the changes in intensity profile along the tree, the central reflex (a central, thin, bright reflection appearing sometimes along the centerline of large vessels, especially arteries) is automatically provided.

Fig. 5.
figure 5

Comparison between real fundus images from the GoDARTS dataset (a, b) and two complete synthetic retinal fundus images generated by our method (c, d).

Full size image

As can be seen in Fig. 6, the bifurcation probability maps of our synthetic images are plausible and similar to the one of the real images.

Fig. 6.
figure 6

Density Map distribution of bifurcation points in synthetic images for artery (a) and vein (b).

Full size image

Our method is fully capable of modelling any biological plausible patterns and outliers given examples of such patterns in the training images. Moreover, the proposed method is potentially capable of dealing with pathologies without increasing its complexity. Indeed, our models can learn their parameters from training set of pathological images.

In absence of quantitative quality criteria, we performed a simple qualitative assessment by asking 7 experts (ophthalmologists and researchers in retinal image analysis) to score the degree of realism of 12 synthetic retinal fundus images, using a scale from 1 to 4, where 1 = not realistic at all, 2 = slightly realistic, 3 = nearly realistic, 4 = very realistic. Scores are summarized in Table 1.

Table 1. Qualitative assessment: scores given by the experts for each image
Full size table

The best image obtained a score of 2.79, while the average score over all the images is 2.13. We did not ask the experts to make any allowance for the fact that many characteristics of fundus images are not modelled (e.g. small capillaries, the vascular network inside the OD). Considering this, the scores suggest that our synthetic images are plausible, as far as the only features generated go. The experts also suggested some improvements: the density of the vessels in some zone is too high, the largest vessels occasionally end abruptly, some first level branches appear too straight and the direction of growth sometimes recoils. These aspects will be considered in our future work. The comments given by the experts will be analysed systematically and properly addressed. A qualitative evaluation on a larger dataset will be used for this scope.

The main purpose of this project was to generate a synthetic dataset providing GT for validation of retinal image analysis algorithms, e.g. vasculature and landmark detection. To demonstrate the suitability of our dataset for this purpose, we compared the performance of a retinal vessel segmentation software, VAMPIRE [17], when run on our synthetic images and on real images. We used 10 images of healthy eyes with manual GT from the HRF dataset, and 10 of our synthetic images including their synthetic binary maps. Segmentation results are evaluated in term of the standard statistical criteria [16]. The comparison of these 2 experiments, summarized in Table 2, shows that our synthetic images behave comparably with real ones in term of vasculature segmentation and certainly in line with the performance of algorithms reported recently in the literature [6]. We note generally small differences between all values.

Table 2. Performance comparison of VAMPIRE segmentation algorithm on real (HRF) and synthetic images: True Positive Rate (TPR), False Positive Rate (FPR), Specificity (Sp), Accuracy (Acc) (mean ± standard deviation).
Full size table

4 Conclusions

This paper has presented a novel technique to generate a reliable synthetic retinal vasculature, as part of an ongoing project aimed to generate full, realistic, synthetic fundus camera images. The results are promising for both the morphology and the texture of the vessel networks. To our best knowledge no similar method has been reported in literature. The encouraging quality of our initial results is supported, so far, by small-scale visual inspection and quantitative experiments. Further improvements to this preliminary work will take into account further properties of real fundus images, including the geometric interaction between arteries and veins, the way vessels radiate from the OD, the vascular network inside the OD, and the appearance of further structures like small capillaries and the retinal nerve fibre layer. Another future direction will be the simulation of lesions.