Keywords

1 Introduction to B-MODE Imaging

B-MODE imaging is one of the most important modes for diagnostic ultrasound. A B-image is represented as a two-dimensional image in grayscale. Each pixel P[x;y] of digitized B-image is displayed as a point with a brightness degree H corresponding to tissue echogenicity. The echogenicity is defined as the ability to reflect or transmit ultrasound waves in the context of surrounding tissues [1]. There are anechoic, hypoechoic and hyperechoic structures. In the case of the reflection is zero, the structure is anechoic. Hyperechoic structures in B-image are displayed as white, hypoechoic as gray and anechoic as black, i.e. each echogenicity grade is expressed by gray level in B-image. Some structures have variable echogenicity according to many factors. The Fig. 1 demonstrates different echogenicity degrees in B-image.

In this study are used images with 256\(^\circ \) of H, in other words from H = 0 to H = 255.

2 A Set of Used Images

This study is primarily focused on analysis of transcranial B-images in transverse thalamic plane [2]. The first investigated structure is the substantia nigra in the midbrain followed by different brain structures commonly examined in neurosonology. Within a pilot study are also used B-images of atherosclerotic plaques. The main goal is to demonstrate using the approach for different B-images. No image pre-processing steps have been used.

Fig. 1.
figure 1

A B-image in transverse thalamic plane with marked window \(5\times 5\) cm consists ipsilateral and contralateral SN

Full size image

3 Principle of the ROI-Based Image Analysis in B-MODE

There are many ways to automatic recognition of structures in the field of digitized image processing. Our study is focused on neurosonology. The substantia nigra is an area in the midbrain. In B-images is characterized by different echogenicity grade depending of death of dopaminergic cells. In normal case the SN is anechogenic, increased echogenicity is caused by the death of the cells which is the main feature of Parkinson’s Disease. Thanks to early diagnostics is possible to help prevent the development of symptoms. ROI-based principle is based on fact that is not required to analyze the whole image but only a ROI which is a subset of the image matrix. For example, ultrasound images used in this study have the resolution of \(768 \times 576\) pixels. In practice, not only in medical image analysis, is needed to observe only examined parts of the image. Formally, ROI is represented as a submatrix of the image matrix. In neurology we need to define ROI for different brain structures for which the software is adaptable. The adaptability is one of the key features to define a new ROI to extend the functionality for different structure in B-image. In the developed application is able to automatically locate the ROI according to coordinate preset. In case of using images with the equal resolution, is able to set location of the ROI with option of manual shift to correct position. Also has been inspected how to recognize the shape and position of substantia nigra. For this purpose has been used a set of 100 images with identical resolution from the same ultrasound machine. The automatic positioning based on artificial neural network with supervised learning have already published [3, 4].

4 Supervised Learning of Artificial Neural Networks in Digital Image Processing

Artificial Neural Networks represent one of phenomena in various image processing fields. In this study, the ANN are used to find position of the ROI according to coordinates. To this processing is used a feedforward MLP network with supervised learning based on Error-Backpropagation Algorithm. The goal is to minimize the network error

$$\begin{aligned} E_t=\sum _{j=1}^{n}{E_p} \end{aligned}$$
(1)

represented as the summation of partial errors

$$\begin{aligned} E_p=\frac{1}{2}(y_j-d_j)^{2} \end{aligned}$$
(2)

where \(y_j\) is j-th output and \(d_j\) is a desired output. The partial errors are computed for each input \(x_j\). The backpropagation algorithm can be described by the following algorithm:

  1. 1.

    Weights initialization to random small values

  2. 2.

    Set the input vector and the vector of desired outputs

  3. 3.

    Compute the real output \(y_j\) for input \(x_j\)

  4. 4.

    Compute the network error (1) from all partial errors (2)

  5. 5.

    Weights adaptation

The algorithm ends if \(E_t < E_d\) or determined number of cycles or maximal time of the learning is exceeded.

Fig. 2.
figure 2

Experimental results of SN positioning by the ANN learning

Full size image

4.1 Training of the ANN and Experimental Results

The goal of the training is to learn correct position of the SN depending on the training set. Because is used the set of images with equal resolution and each image is cut in the same size, the correct position of the SN can be predefined. Used ANN is composed of 2 hidden layers. Thanks to equal resolution and window size, the default correct position of the center of the ROI is given by

$$\begin{aligned} S=\left[ 2.00;2.00\right] \end{aligned}$$
(3)

as the initial position of desired output (5). If the output position is incorrect, is necessary to retrain the network by manual positioning. The coordinates of a new position are added into the training set. The goal is to achieve successful detection of the substantia nigra area by minimization of the error of the correct position. See Fig. 2 with experimental results of the ROI position. Totally has been used a set of 100 images to training the ANN with correct position. To train of the ANN have been used images with normal, e.g. low echogenicity grade of SN. It was observed if the results are reliable used also for hyperechoic SN (Fig. 2). The finding of the position can be successfully used for images with equal resolution and in which SN. The ANN should be better adapted with many more samples with significant differences between images. The adaptation of the position finding is the one of crucial goals of the ANN training. In other words, the goal is to train the ANN to reliable recognition of the position of the SN for any B-image in transverse thalamic plane (Fig. 1). There are two following aspects:

  • minimal total error \(E_t\)

  • to find a position which is applicable for almost all B-images in thalamic transverse plane

There are two crucial requirements to successful application. In experimental study with a set of 100 images which have equal resolution and size, the ANN evinced acceptable results. Correlation coefficient between desired and observed results was \(r>0.8\) and also have been judged intra- and inter- \(\kappa \) coefficients in which were achieved \(\kappa _{intra}=0.75\) and \(\kappa _{inter}=0.82\).

4.2 Brief Description of Used ANN

In this case the following ANN is used:

  • multilayer feedforward network

  • 1 input layer, two hidden layers, 1 output layer

  • supervised learning based on Error Back-Propagation algorithm

  • logic sigmoid activation function

To input layer is get an input vector and a weight vector of n inputs. Based on multiplication is computed the inner potential

$$\begin{aligned} \zeta =x_{j}\cdot w_{j} \end{aligned}$$
(4)

where \(x_i\) are inputs of coordinates and \(w_i\) are randomly set weights from 0 to 1. Each input is multiplicated by the weight. In this case the inputs represents the coordinates S (3). During the computation in hidden layers is computed the partial error \(E_p\) (2) for each input. Log-sigmoid transfer function

$$\begin{aligned} y=\frac{1}{1+e^{-\lambda x}} \end{aligned}$$
(5)

and

$$\begin{aligned} y=\frac{1}{1+e^{-x}} \end{aligned}$$
(6)

if the \(\lambda =1\) (sigmoid steepness). The sigmoid steepness influences learning speed. The goal is to learn ANN to compute optimal coordinates of the position of the SN which can be applicable in general. To global error (2) minimization is used Mean Squared Error (MSE) given by

$$\begin{aligned} \frac{1}{n}\sum _{j=1}^{n}\left( y_j-d_j \right) ^2 \end{aligned}$$
(7)

In general, MSE is considered as the error between predictions and observed values. In this case, MSE describes the error between desired coordinates and observed coordinates of the position. The goal is to find the coordinates with minimal error and which can be used in general for any B-image.

4.3 Complexity of Intelligent Learning

Intelligent learning of ANN is a complex and time-consuming problem. In case of B-images, the learning is very complicated due to B-image structure. There are many aspects which make the image processing very complicated, e.g. nonlinear speckle noise and ultrasound artifacts (shadowing, reverberation, mirror imaging), etc. The accuracy \({>}\)95% of the learning is required by an experienced sonographer. From one’s own experience and from the point of view of the experienced sonographer, the learning is not the main goal to useful employment. The goal of this research is to use many different ANN approaches. In other words, we can find the optimal activation function, number of hidden layers, weights adaptation and another criteria for an optimal ANN learning. The learning phase is the first phase of the process. Accuracy of the learning phase must be validated by evaluation phase; the comparison of the global error (2) of the ANN. There is highly time-consuming phase with repeating of the learning phase due to complexity of the B-image structure. There is necessity to use a large set of images with different resolution, SN echogenicity grade and position of the SN to evaluate the accuracy.

5 Using in Developed Software

We have developed the software tool designed for evaluation of echogenicity grade of the SN [5, 6]. Five echogenicity grades are distinguished in neurosonology, i.e. from hypoechoic to hyperechoic. This software is based on binary thresholding algorithm inside the selected ROI and computing echogenic area inside the SN to Parkinson’s Disease diagnosis [6]. Greater echogenic area represents higher probability of Parkinson’s Disease. Let H represents the intensity level of a pixel and T is a threshold, then

$$\begin{aligned} H>T \end{aligned}$$
(8)

For each T is computed how many pixels are \({>}T\) and subsequently is converted into real \(\mathrm{mm}^2\). The echogenicity grade of the SN is evaluated on the basis of number white pixels depending on the condition (4) for thresholds \(T \in \langle 0;255 \rangle \). See Fig. 3 with an example of decreasing number of pixels depending on threshold T.

Fig. 3.
figure 3

Thresholding results for SN measurement.

Full size image

Output values, their context and statistical analysis of measured descriptors is discussed in [7,8,9].

6 Prospective Study on Atherosclerosis Recognition

The methodology has been developed with respect to universal using not only for transcranial images. The software could be used for various ultrasound B-images because the principle is designed with respect to universal application. We want to investigate risk evaluation of atherosclerotic plaques in B-images. Firstly, we must find some features in B-images for reliable detection of atherosclerosis. The goal of this prospective study is to inspect how atherosclerotic plaques are distinguishable. Atherosclerotic plaques can be distinguished by size, homogeneity, shape, composition, etc. This prospective study is focused on finding these features to distinguish normal and pathological cases. In comparison with examined brain structures, in case of atherosclerotic plaque is not applicable to use a predefined ROI with equal size and shape. There are two main known crucial limitations. The first limitation is about displaying of atherosclerotic plaques in B-image; how to reliably distinguish plaques. The second limitation is closely related with the principle of the developed software. The predefined ROI is used in case of analysis of SN and some other structures which substantially do not change size and position. The output from the software represents measured values of the echogenic area. Due to using the equal ROI, echogenicity grade can be compared depending on area inside the ROI. To investigation of SN echogenicity grade is used an elliptical ROI with default area A = 50 mm2, see Fig. 3. In case of investigation of atherosclerotic plaques is not applicable to use ROI with constant area. Each plaque appears differently and must be defined a special ROI and we cannot compare decreasing area (Fig. 3). Figure 4 shows three different ROIs corresponding to atherosclerotic plaques defined by an experienced sonographer.

Fig. 4.
figure 4

Three different ROI corresponding to atherosclerotic plaques in B-image

Full size image

Due to different size of ROI in each case, is not possible to compare decreasing area, see Fig. 5. In case of SN and another brain structures is used equal size of the ROI for all cases.

Fig. 5.
figure 5

The results of descending area are incomparable

Full size image

6.1 Automatic Recognition of the Plaque Using ANN

For atherosclerotic plaques using ANN is much more complicated in comparison with SN. There is a way how to recognize shape of the plaque using ANN. The automatic recognition of the shape could be based on Region-Based training or training based on echogenicity grade. There are two crucial barriers. Firstly, the learning of the ANN to successful recognition of the plaque shape could be time-consuming problem because the shape has no standardization. So, to train the ANN requires a huge sample of different normal and pathological findings, e.g. based on correlation with histology. Secondly, how to select some features as inputs into ANN input layer to computing? Simply, what we get from one image as reliable feature could not be reliable from another image. The recognition of the plaque by ANN is much more complicated in comparison with finding position of the SN. In B-image each plaque is displayed differently (different size, shape, echogenicity, etc.). On the ground of the differences the recognition of the plaque is highly time-consuming. First of all, very large set of images is needed to start. We have only small set of images at present but a large database is created concurrently with analysis of preserved histological patterns. To find features, the patterns must be analyzed and compared with B-images. After this step could be analyzed if findings in B-image correspond to histological patterns.

6.2 Ratio of Hyperechoic Pixels

In connection with these facts is possible to compute ratio instead of area. The core of the computing could be based on ratio between number of hyperechoic pixels to total number of pixels independently on shape and size of the ROI. The ratio could be a feature to distinguish normal and pathological cases. Is this solution acceptable for medical practice? Exist some better features to atherosclerotic plaques analysis in B-images? Even though is one of possible ways to analyze atherosclerotic plaques in B-images with using developed software. It is to research in the near future. The software has been also used for experimental investigation of insula [10]. Recently has been performed an experimental study with 23 images of atherosclerotic plaques [11]. The images were visually distinguished (homogeneity, low heterogeneity and strong heterogeneity) by an experienced sonographer. The aim was to observe reproducibility using selected statistical descriptors. However, the results did not produce reliable markers caused by number of images and the ambiguity of manually selected ROI.

7 Results, Conclusions and Future Goals

The study is focused on ROI-based image processing of ultrasound B-image on the basis of developed software in MATLAB originally designed to analysis of echogenicity of substantia nigra. The software has been developed since 2009 but recently the limitations are crucial to using the software. These limitations lie in the approach how to define ROI. The future goals are focused on using the software to analysis of atherosclerotic plaques in B-images. Investigation of automatic finding of the ROI using ANN is one of crucial part of this research. There are some different ways how to recognize ROI using ANN but is not a primary goal at present. In case of ultrasound B-images, it requires very exact and time-consuming learning of the ANN given the B-image complexity (noise, resolution, initial settings of gain, etc.). There is no large set of images correlated with histological analysis. Nevertheless, the results demonstrated that finding the position of the SN is partially applicable, primarily for a set of the images with equal size and resolution. The achieved correlation was \({>}\)0.8 and inter-/intra- \(\kappa \) coefficients were 0.75 and 0.82. To find an optimal learning and evaluating phase of ANN is needed larger set of images and try to set different input parameters of ANN. Using ANN to automatic detection of the atherosclerotic plaque is considered as very progressive way but also highly time-consuming and much more complicated. The performed experimental study with the set of 23 images is not sufficient to unbiased results. First of all, there is necessity to obtain a large set of B-images compared with histological patterns to find corresponding features depending on progression of atherosclerosis.