Introduction

Every day massive amount of medical images is acquired and preserved as databases in health care centers for future references, disease diagnostics and for proper treatment. Physician uses these large medical image data as the modern frontier of the medical field and improves the quality of medical treatment. Many referring physicians also show a great interest for direct access to medical image data and to discuss with their hospital colleagues by retrieving the required image immediately and have the chance to confer the case collaboratively with other specialists for successful decision making. Healthy people also show interest to have a master medical check up periodically which increases the accumulation of medical images in the hospitals. Hence, there is a need to find the most clinically relevant images in response to specific information required. Content-Based Image Retrieval (CBIR) focused to follow a line of investigation to fulfill this requirement. Where CBIR is a well-known technology used to organize an image database by their features (contents) and to search it according to the requirement. Ultimately, the Content-Based Medical Image Retrieval (CBMIR) attracts extensive attention in the research community by its significant clinical benefits and the pace of its research is still rising.

Duncan and Ayache listed out four important challenges should be faced while analysing medical images [1]. They are: (1) addressing the biological problems, (2) Characterizing different imaging modalities, (3) identifying proper techniques for geometrical analysis of both normal and abnormal information, and (4) integrating with other communities’ research activities to provide new opportunities. Lakovidis et al. [2] extracted multi-scale statistical features using a 2D discrete wavelet transform and by clustering those features into small patterns, by comprising the sets of these smaller patterns and representing as complex patterns. Rahman et al. [3] proposed a multimodal query expansion method by integrating textual keywords and visual data. Bag-of-features [4] such as Scale Invariant Feature Transform (SIFT) and image patches have been modelled using the quadratic programming assignment and analyzed by both the strong and weak classifiers on the ImageCLEF, CT and the basal-cell carcinoma datasets by Wang et al. in 2011. Followed by, Quellec et al. [5] introduced a content-based heterogeneous information retrieval framework using a Bayesian network. Then Cao et al. [6] proposed a multimodal medical image retrieval approach using the statistical graphic model and deep learning. To overcome the limitation of semantic gap, the authors introduced an extended probabilistic latent semantic analysis model by combining both the visual and textual information from medical images for cancer care. And to derive the missing modality, they developed a Boltzmann machine-based multimodal learning model. Next, saliency based folding method has been developed by employing Local Binary Patterns (LBP) and Support Vector Machines (SVM) in 2015 [7]. Albarqouni et al. [8] used Convolutional Neural Networks (CNN) through additional crowd sourcing layer (AggNet) for deep learning for Mitosis detection from the breast cancer histology images. Later, fuzzy aggregation operators based dental diagnosis has been performed dental X-ray images [9]. Shortly, Speeded Up Robust Feature (SURF) descriptor has been developed on the lung images for implementing an efficient CBMIR [10]. On the Image Retrieval in Medical Applications (IRMA) dataset, Radon Bar-Code (RBC), Convolutional Neural Network Code (CNNC) and Regions of Interest (ROI) based matching has been investigated along with the cross-correlation method [11]. Recently, Expectation Maximization (EM) and Fuzzy C-Means (FCM) were implemented on MRI brain images [12].

Feature extraction takes a vital role of extracting the fine features of images for better image retrieval. The efficiency of the retrieval can be augmented by choosing the perfect feature extraction technique. Furthermore, the features present in the medical images are complex; anatomical structures are highly irregular and multimodality image’ structures are misaligned. To face this challenge, a novel texture based local pattern descriptor has been proposed in this paper to extract features from MRI brain images and to retrieve them automatically whenever needed.

The evolution of local patterns is as follows: the Local Binary Pattern (LBP) [13] has been initially intended for texture description. Subsequently, Tan and Triggs [14] provided the Local Ternary Pattern (LTP) as an expansion of the LBP. Later, Murala et al. proposed the Local Tetra Pattern (LTrP) [15] for facial image recognition and the Directional Binary Wavelet Pattern (DBWP) [16] for MRI, CT and facial image retrieval. After that, Murala and Wu proposed the Peak Valley Edge Patterns [17] and the Local Mesh Pattern (LMeP) [18] on biomedical images. Later, the Local Vector Pattern (LVP) [19] has been developed for facial feature extraction. Then Yesubai and Ravi proposed the Local Mesh Co-occurrence Pattern (LMCoP) on natural scenery images [20]. Thus, the local patterns have gained much attention in the facial recognition area than the medical field. It stimulates the idea of introducing a new hybrid local pattern LMVCoP for medical image retrieval application.

The proposed method

This section describes the proposed CBMIR based on LMVCoP technique in more detail. Figure 1 depicts the flow of the proposed system. The steps followed in this work are listed below:

Fig. 1
figure 1

Flowchart of the proposed system

Full size image

Step 1: Pre-processing

Given image is first pre-processed to synchronize the content in a database and thus create effective feature extraction from the images based on the same basis. The image is initially converted into a gray-scale image and resized into 256 × 256 pixels. The image is then normalized using contrast enhancement of adaptive histogram equalization. Since adaptive histogram equalization distributes the most repeated intensity values and transforms the lower local contrast into higher contrast effectively, this method is used here. Followed by, the process of smoothing that reduces the noise was performed to produce a more accurate intensity surface. Since Gaussian filter is very effective for removing Gaussian noise, reducing edge blurring, rotationally symmetric and computationally efficient for MRI brain images, Gaussian smoothing is used in this work for a removal of small unconnected objects and to adjust for residual between-subject neuro-anatomic differences.

Step 2: Extracting features using local vector co-occurrence pattern

Given a pre-processed local sub-region I, α is the index angle, g c is the center pixel in I, D is the distance between the centre pixel and its neighboring pixels in the α direction. First, a vector that denoted as V α , D (g c ) is calculated by

$$ {V}_{\alpha, D}\left({g}_c\right)=\left(I\left({g}_{\alpha, D}\right)-I\left({g}_C\right)\right) $$
(1)

Then, the LVP N , R , α (g c ), in the α direction is encoded as

$$ {LVP}_{N,R,\alpha}\left({g}_c\right)=\left\{T\left({V}_{\alpha, D}\left({g}_{1,R}\right),{V}_{\alpha +{45}^{\circ },D}\left({g}_{1,R}\right),{V}_{\alpha, D}\left({g}_c\right),{V}_{\alpha +{45}^{\circ },D}\left({g}_c\right)\right),\dots, T\left({V}_{\alpha, D}\left({g}_{N,R}\right),{V}_{\alpha +{45}^{\circ },D}\left({g}_{N,R}\right),{V}_{\alpha, D}\left({g}_c\right),{V}_{\alpha +{45}^{\circ },D}\left({g}_c\right)\right)\right\} $$
(2)

Where N = 1 , 2 , … , M ; R = 1,

T (·, ·) is defined as

$$ T\left({V}_{\alpha, D}\left({g}_{N,R}\right),{V}_{\alpha +45{}^{\circ},D}\left({g}_{N,R}\right),{V}_{\alpha, D}\left({g}_c\right),{V}_{\alpha +45{}^{\circ},D}\left({g}_c\right)\right)=\left\{\begin{array}{cc}1,& if\ {V}_{\alpha +45{}^{\circ},D}\left({g}_{N,R}\right)-\left(\frac{V_{\alpha +45{}^{\circ},D}\left({g}_c\right)}{V_{\alpha, D}\left({g}_c\right)}\times {V}_{\alpha, D}\left({g}_{N,R}\right)\right)\ge 0\\ {}0,& else\kern21em \end{array}\right. $$
(3)

Next, the LVP N , R (g c ), is computed by combining four 8-bit binary patterns. For every center pixel of I, a 32-bit binary pattern is generated.

$$ {LVP}_{N,R}\left({g}_c\right)={LVP}_{N,R,\alpha}\left({g}_c\right),\left|\alpha ={0}^{{}^{\circ}},{45}^{{}^{\circ}},{90}^{{}^{\circ}},{135}^{{}^{\circ}}\right. $$
(4)

Finally, the resultant LVP map is computed using GLCM, where, M is a matrix whose size M x × M y is equal to the number of gray-levels 0 to M-1. At an angle θ and d distance, the variation between gray level i and j, has been calculated and represented as the matrix element C(i,j|d,θ). C x (i) and C y (j) are computed by

$$ {C}_x(i)={\sum}_{i=0}^{M_x-1}C\left(i,j\right) $$
(5)
$$ {C}_y(j)={\sum}_{j=0}^{M_y-1}C\left(i,j\right) $$
(6)

Once the Co-occurrence matrix has been formed, the following Haralick texture features can be computed by

$$ CONTRAST={\sum}_{I=0}^{m-1}{n}^2\left\{{\sum}_{j=0}^{M_x-1}{\sum}_{j=0}^{M_y-1}C\left(i,j\right)\right\} $$
(7)

where n = |i − j| , i ≠ j

$$ CORRELATION={\sum}_{i=0}^{M_x-1}{\sum}_{j=0}^{M_y-1}\frac{\left\{i\times j\right\}\times C\left(i,j\right)-\left\{{\mu}_x\times {\mu}_y\right\}}{\sigma_x\times {\sigma}_y} $$
(8)

where mean μ x  and μ y  take the average level of intensity of given input such as

$$ {\mu}_x={\sum}_{i=0}^{M_x-1}{iC}_x(i) $$
(9)
$$ {\mu}_y={\sum}_{j=0}^{M_y-1}{jC}_y(j) $$
(10)

whereas the variance σ x  and σ y  describe the variation of intensity around the mean.

$$ Energy={\sum}_{i=0}^{M_x-1}{\sum}_{j=0}^{M_y-1}{C}^2\left(i,j\right) $$
(11)
$$ Homogeneity={\sum}_{i=0}^{M_x-1}{\sum}_{j=0}^{M_y-1}\frac{C\left(i,j\right)}{1+\left|i-j\right|} $$
(12)

Step 3: Extracting features using local mesh co-occurrence pattern

Given a pre-processed local sub-region I, the relationship among the surrounding neighbours of the centre pixel I 1(g n ) is computed by

$$ {I}^1\left({g}_n\right)=I\left({g}_m\right)-I\left({g}_n\right) $$
(13)

where n = 1 , 2 , … , 8. Then the LMeP can be calculated by

$$ {LMeP}_{N,R}^M={\sum}_{n=1}^N{2}^{\left(n-1\right)}\times {f}_1\left({g}_m-{g}_n\right) $$
(14)
$$ {\displaystyle \begin{array}{c}\hfill m=1+\mathit{\operatorname{mod}}\left(\left(n+N+M-1\right),N\right)\hfill \\ {}\hfill \forall M=1,2,\dots, \left(\raisebox{1ex}{$N$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\right)\hfill \end{array}} $$
(15)

After computing the LMeP, apply GLCM on the resultant LMeP map using Eqs. (5–12).

Step 4: Computing local mesh vector co-occurrence pattern

The local pattern results are combined using the sum rule. For Example, LMCoP denotes GLCM of LMeP, similarly LVCoP denotes GLCM of LVP, and then the combined approach of these two patterns is abbreviated as LMVCoP, which can be written as:

$$ LMVCoP= wLMCoP+\left(1-w\right) LVCoP $$
(16)

where, w is a weighted parameter that ranges from 0.1 to 0.9. The relationship between LMCoP and LVCoP can be investigated by changing w, and the performance of LMVCoP can also be evaluated. In this work, initially, all the tests were performed for w = 0.2

Step 5: Retrieval

Once the proposed model LMVCoP is trained for all the medical images of the given dataset, the feature vectors are stored as a feature dataset with reduced dimension. Then, the step, retrieval is performed. The similarity measurement is computed between the feature vectors of feature dataset and the feature vector of query image based on Squared Euclidean distance and sorted the values in ascending order. The image of the database which has very minimum difference or shorter distance attains a higher position and considered as the image having higher similarity. As per request from the user the retrieved images have been displayed in the main Graphical User Interface (GUI).

Step 6: Performance evaluation

The effectiveness of the LMCoP and LVCoP techniques is evaluated using the following error sensitivity measures:

Mean Squared Error:

$$ MSE\left(I,E\right)=\frac{1}{MN}{\sum}_{i=1}^M{\sum}_{j=1}^N{\left({E}_{i,j}-{I}_{i,j}\right)}^2 $$
(17)

Peak Signal-to-Noise Ratio:

$$ PSNR\ \left(I,E\right)=10\ \mathit{\log}\left(\frac{\mathit{\max}\Big(\left.{I}^2\right)}{MSE\left(I,E\right)}\right) $$
(18)

Signal-to-Noise Ratio:

$$ SNR\left(I,E\right)=10\;\log \left(\frac{\sum_{i=1}^M{\sum}_{j=1}^N{\left({I}_{i,j}\right)}^2}{M.N. MSE\left(I,E\right)}\right) $$
(19)

Average Difference:

$$ AD\left(I,E\right)=\frac{1}{MN}{\sum}_{i=1}^M{\sum}_{j=1}^N\left({I}_{i,j}-{E}_{i,j}\right) $$
(20)

Normalized Absolute Error:

$$ NAE\left(I,E\right)=\frac{\sum_{i=1}^M{\sum}_{j=1}^N\left|{I}_{i,j}-{E}_{i,j}\right|}{\sum_{i=1}^M{\sum}_{j=1}^N\left|{I}_{i,j}\right|} $$
(21)

Normalized Cross-Correlation:

$$ NXC\left(I,E\right)=\frac{\sum_{i=1}^M{\sum}_{j=1}^N\left({I}_{i,j}.{E}_{i,j}\right)}{\sum_{i=1}^M{\sum}_{j=1}^N{\left({I}_{i,j}\right)}^2} $$
(22)

where, I is a given input image. E is the estimated or output image. M × N is the size of the image. The efficiency of the proposed image retrieval system is evaluated using the following performance evaluation metrics: Precision P j (M) is calculated by

$$ {P}_j(M)={I}_M/M $$
(23)

Whereas Recall, R j (M) is computed by

$$ {R}_j(M)={I}_M/S $$
(24)

Consider j th dataset image is used as a query image. I M is the number of relevant images retrieved, M is the total number of images retrieved and S is the total number of relevant images. The average precision \( {P}_a^i(M) \) and average recall \( {R}_a^i(M) \)for each class in the database are computed by

$$ {P}_a^i(M)=\frac{1}{S}{\sum}_{j=1}^S{P}_j(M) $$
(25)
$$ {R}_a^i(M)=\frac{1}{S}{\sum}_{j=1}^S{R}_j(M) $$
(26)

where i is the class number and a is the average. The Average Retrieval Precision (ARP) and Average Retrieval Rate (ARR) respectively were calculated using Eqs. (27 and 28).

$$ {P}_a^T(M)=\frac{1}{C}{\sum}_{i=1}^C{P}_a^i(M) $$
(27)
$$ {R}_a^T(M)=\frac{1}{C}{\sum}_{i=1}^C{R}_a^i(M) $$
(28)

where C is the total number of classes in the database. In this work, since the proposed LMVCoP is investigated only in MRI brain images, C = 1, Hence \( {P}_a^i(M)={P}_a^T(M) \) and \( {R}_a^i(M)={R}_a^T(M). \)

Results and discussion

The proposed LMVCoP is implemented in MATLAB software by building GUI in an Intel (R) Core i3 processor, 3.20 GHz, 4 GB RAM computer with the Microsoft Window7 Professional operation system platform. To test the efficiency of the proposed methodology a large data set of OASIS: MRI images [21] have been used in this work. There are four groups of dataset contain 106, 89, 102, and 124 MRI brain images were used for the test. For a data set, the evaluation is performed independently.

The proposed system interacts with the user through a succession of GUIs. First, in the pre-processing step, the given query image is converted into gray-scale image. After that, the scaling algorithm has been applied to resize the gray-scale image. Then, the image features have been enhanced with the help of contrast enhancement algorithm. Next, Gaussian smoothing is used to reduce the noise. Second, the features are extracted using the LVP and GLCM. Finally, LVCoP map is obtained. The results are shown as a snapshot in Fig. 2. The validation has been done using error sensitivity measures and values are displayed. After computing LVCoP, the function calls back to the main menu and the computation of LMCoP carries out as per the above-mentioned procedure. The results of LMCoP are shown in Fig. 3. The fusion of LVCoP and LMCoP are done and the LMVCoP features are generated and stored as feature vector database. The similarity between the features of database images and the query image are performed using Squared Euclidean distance and the retrieval images are displayed to the user. A screen shot of the results of top 10 retrieved images are shown in Fig. 4. Further, more number of (20 and 30) images are retrieved and displayed. A screen shot of the results of the various number of images retrieved are shown in Figs. 5 and 6 respectively.

Fig. 2
figure 2

The GUI for computing LVCoP

Full size image
Fig. 3
figure 3

The GUI for extracting features using LMCoP

Full size image
Fig. 4
figure 4

The main GUI of proposed LMVCoP system showing the step-wise results and the first 10 retrieved MRI brain images

Full size image
Fig. 5
figure 5

A screen shot of retrieving top 20 images

Full size image
Fig. 6
figure 6

A screen shot of retrieving top 30 images

Full size image

The performance of LMCoP and LVCoP has been compared for a given input brain MRI image in terms of error sensitivity measures such as Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), Signal-to-Noise Ratio (SNR), Average Difference (AD), Normalized Absolute Error (NAE) and Normalized Cross Correlation (NXC) and the results are listed out in Table 1. When MSE is low, the PSNR value is high, then the quality rate turn into high. Here, PSNR is in the range of 50-60db. Hence, it is understood that the quality of processed image is high. Generally, SNR is defined as the power ratio and expressed in db. But, here it is expressed as the signal ratio. Hence, it is given in number. In general, there are 17 Haralick features. Out of 17 features, the Contrast, Correlation, Energy and Homogeneity features are considered as very important features that are obtained by Eqs, (7, 8, 11 and 12) and recorded in the Table 2. The contrast values reflect the sharp structural variations of medical images. The correlation imply that fine linear dependent features are present in the given images. The energy is used to measure texture crudeness; if it is equal for all the images, then it is concluded that the co-occurrence matrix has same values. But, from the values in Table 2, it is understood that the values of co-occurrence matrix are unequal. In view of the fact that the values of homogeneity of images #1, #5, #10 are nearer to 1 illustrate that they have diagonal GLCM. The Table 3 listed out the time spent for performing feature extraction and retrieval processes using the LMVCoP and the existing methods of LTCoP, PVEP, LMeP LBDP and LVP on bio-medical images. From Table 3, it is understood that the speed of feature extraction using LMVCoP is faster than the existing methods. Table 4 depicted the group-wise ARP based retrieval performance of all the state-of-the-arts of the LPD on the OASIS: MRI brain dataset. The comparison Table 4 depicts that the the LMVCoP outperforms in groups 1, 2 and 3. In group 4, GLTCoP, GLTCoPu2, LDMaMEP and LMeCH perform well than the proposed LMVCoP descriptor.

Table 1 Validation of LMCoP and LVCoP using Error Sensitivity measures
Full size table
Table 2 Haralick Features for the sample MRI brain Image based on LMVCoP
Full size table
Table 3 Time spent for Feature Extraction (FE) and Image Retrieval (IR) in seconds for OASIS-MRI database
Full size table
Table 4 ARP based retrieval performance of all the methods on OASIS database (Group- wise)
Full size table

Figure 7 illustrates the performance of LMVCoP on OASIS: MRI brain images in terms of the ARP based on Squared Euclidean distance. It is evident that the average retrieval precision of LMVCoP, depicted in Fig. 7 is higher than all other existing methods. The high precision values obtained indicate that the LMVCoP is able to retrieve more relevant images. The ARR result exposed in Fig. 8 illustrates that as precision increases, the recall value decreases. Hence, it is known that the LMVCoP has given better performance. The proposed LMVCoP descriptor achieved the ARP of 87.57% and the ARR of 53.21% for the first 10 number of retrieval images.

Fig. 7
figure 7

The retrieval performances of the LMVCoP compared to LTCoP, PVEP, LMeP, LBDP and LVP methods in terms of ARP

Full size image
Fig. 8
figure 8

The retrieval performances of the LMVCoP compared to LTCoP, PVEP, LMeP, LBDP and LVP methods in terms of ARR

Full size image

Conclusions

In this paper, the LMVCoP descriptor has been proposed to obtain better retrieval performance on medical images for the fast decision making in clinical application. The LMVCoP extracted the texture features of the MRI brain images by concatenating the LPD and GLCM. The performance of LMVCoP has been compared in terms of performance evaluation metrics with the existing descriptors that were used for biomedical indexing and retrieval such as LTCoP, PVEP, LMeP, LBDP and LVP. LMVCoP descriptor achieves 87.57% of ARP and 53.21% of ARR. The time spent by LMVCoP descriptors for feature extraction is 04.29 s and retrieval is 0.21 s that were also less. The simulation results and investigations prove that the LMVCoP outperforms the LTCoP by 31.61%, the PVEP by 17.43%, the LMeP by 6.21%, the LBDP by 4.6%, and the LVP by 4.05% in terms of ARP. Implementing higher-order LMVCoP is an immediate challenge. In future, it is planned to implement higher order LMVCoP and to combine LPDs and neural network methods for retrieving large-scale high dimentional image datasets.