Introduction

Medical Imaging

Medical image processing is the technology used as a non-invasive technique to capture the images of a human body by medical professionals for identifying the disease or diagnosis of the disease. It is indirectly helping the medical professionals or doctors to detect abnormalities or lesions occurred in human body in faster and innovative way. Medical imaging is not only referred as an imaging technique but have become a standard for detection of many diseases related to diabetes, cancer, neurological conditions and cardiovascular diseases like heart failure, cardiac arrest etc. [1].

Diabetes Mellitus

Diabetes Mellitus caused due to the defect in insulin production which leads to long term harm or breakdown of organs such as eyes, kidneys, nerves, heart, blood vessels etc. [2]. Diabetes is classified into two types, type 1 diabetes where body’s own resistant framework attacks the pancreases which leads to pancreases unable to produce the insulin, this type of diabetes is mostly observed in adults and obese people [3]. In type 2 diabetes resistant framework of diabetic patient kills beta cells which are responsible for production of insulin in body, this type of diabetes is commonly observed in children [3]. As per the statistics given by International Diabetes Federation (IDF) Diabetes Atlas 2019 around 463 million adults are currently suffering from diabetes and out of which 79% of individuals live in low and center pay nations. India is 2nd ranked country where currently 77 million individuals are experiencing diabetes and this count will go to 134.2 million continuously 2045 [4]. IDF Diabetes Atlas 2019 report also gives information about Diabetic Eye Disease (DED) which is considered as a complicated condition of the diabetes and leads to Diabetic Retinopathy (DR), Diabetic Macular Edema, Double vision and inability to focus [4, 5].

In recent years, people suffering from diabetes have exponentially increased. Diabetic patients are more prone to suffer from DR and it is being observed that there is 6.2% possibility to develop into Proliferative DR in a year [6]. DR has become the biggest research area for researchers as people can lose their complete vision leading to blindness and it is very important to develop an Automatic Computer Aided Diagnosis (CAD) system for detection and grading of DR and DME in their early stages.

Main cause of visual impairment in working age people is observed due to Diabetic Retinopathy (DR), however early findings and regular therapy can prevent eye sight and complete vision loss [4]. In the traditional or manual method of detection of DR, retinal images of eye are taken and analysis and explanation is done by eye specialist which is tedious and overpriced as well but the biggest concern is the proportion of the patients to ophthalmologist which is very less in the low and center pay nations like India [4]. The other method for the detection of DR is automated system where Artificial intelligence(AI),Deep Learning(DL) with advance image processing techniques can be used for the detection of DR above the traditional method and results can be obtained in less time with minimum cost [7]. Automated processing technique have been playing most promising role in the analysis of DR. Deep learning algorithms such as convolution neural networks have become one of the important choice in DR medical imaging analysis which helps in identifying if patients needs further checkup or not. Also Machine Learning (ML) algorithms are used for the segmentation, classification of objects or lesions, region and landmarks, localization, content based image retrieval [8].

DR is categorized Non-proliferative (NPDR) in which injuries such as Micro aneurysms (MAs), Hemorrhages (HAs), exudates or Proliferative (PDR) where neovascularization (NV) or vitreous hemorrhages are seen whereas in Diabetic Macular Edema (DME) retinal thickening or hard exudates are observed [9].

Anatomy of Human Eye and Clinical Features of DR: Human eye structure comprises of various cellular structure tissues which help for maintaining and functioning of a vision [10]. Retina assumes a significant part in vision system, the properties and the functioning of the retina helps to find out the changes occurred due to the eye diseases and fundus retinal imaging is generally utilized imaging method for capturing the auxiliary features of retina [11]. The central area within the retina is known as macula consisting of fovea, rods & cones responsible for photographic vision [10]. Below shown Fig. 1 shows the fundus image of healthy retina with important landmarks and Fig. 2 shows the normal sight with changes occurred in the sight due to DR and DME.

Fig. 1
figure 1

Healthy Retinal fundus image of human eye showing the key landmarks such as retinal veins, fovea, macula, and optic disc (OD) considered in detection of DR [11]

Full size image
Fig. 2
figure 2

a Normal vision b vision affected due to DR c vision affected due to DME [11]

Full size image

Table 1 gives a summary of stages of diabetic retinopathy based on analyzed clinical features and Fig. 3 [12] shows typical fundus images showing stages of DR.

Table 1 Classification or Stages of DR [11, 13]
Full size table
Fig. 3
figure 3

Typical fundus images showing stages of DR with fig. (a) retinal features optic disc, macula, blood vessels fig (b–f) lesions (abnormalities) such as microaneurysms, exudates, hemorrhages [12]

Full size image

The main aim of this literature survey is to review and study different types of retinal imaging techniques used for DR and DME detection, availability of different public and private datasets in the field of diabetic retinopathy, particularly analysis of recent methodologies such as machine learning, deep learning, image processing for detection of DR and DME based on CAD system. Study of performance evaluation metrics used in existing algorithms. Finally, the challenges, future scope and conclusion are covered in the study. This review paper mainly focuses on the recent 2-year research work done in the domain of Diabetic Retinopathy and Diabetic Macular Edema detection or grading and considers studies and algorithms or methodologies published in reputed journals such as Elsevier, IEEE, Springer, ACM, PubMed etc.

Retinal Imaging Techniques

Retinal imaging techniques have been playing vital role in medical care with management in retinal diseases and have rapidly achieved advancement in last 160 years [14]. Fundus photography is the representation of 3D structured retina onto 2D plane where color filters, fluorescein dyes are used to capture the fundus images [12]. Below is the list of few imaging techniques.

  1. 1.

    Fundus Imaging: The image taken utilizing the measure of reflected explicit wave band is called fundus photography and image taken utilizing the measure of reflected Red, Green, Blue (RGB) waveband is called as color fundus photography [14].

  2. 2.

    Scanning Laser Ophthalmoscopy (SLO): SLO is the image taken using the amount of reflected signal wavelength laser light obtained in a time sequence [14].

  3. 3.

    Optical Coherence Tomography (OCT) Imaging: OCT provides cross-sectional information of retinal structure based on the principle of interferometric technique and focuses on the problems like retinal layer segmentation i.e. thickness measurement [15].

  4. 4.

    OCT Angiography (OCTA) Imaging: OCTA is captured using amount of reflection of laser beam on the plane of red corpuscle and gives volumetric blood flow information [16].

  5. 5.

    Wide Field (WF) and Ultra-Wide Field (UWF) Imaging: In normal fundus photography or WF Field of View (FOV) is 20–50 degrees which covers only some of the portion of retina due to which sometimes few features or all features cannot be detected but in UWF imaging FOV is up to 200 degrees which is the most significant advancement for detecting DR, PDR and Retinal Vein Occlusion(RVO) [17, 18].

Publically Available Retinal Images Datasets

There are many retinal image datasets are available publically to the researchers for training, testing and validating purpose with ground truths verified by medical experts. Below is the list of few of the datasets.

  1. 1.

    Digital Retinal Images for Vessel Extraction (DRIVE): This dataset comprises of 33 healthy retina and 7 disease images and all annotations for vessel segmentation is given for all images by medical experts [19].

  2. 2.

    Fundus Image Registration (FIRE): Total number of images 129 with 134 image pairs with corresponding ground truths and binary color region of interest (ROI mask), specially focuses on image registration [19].

  3. 3.

    STructured Analysis of Retina (STARE): It consists of 4 sets of 100 images sums up to total 400 images, annotations for all images are provided also labeling is done on 40 images and 80 images for vessel segmentation and optic nerve head detection respectively [19].

  4. 4.

    High Resolution Fundus image database (HRF): HRF provides 15 images for each healthy, DR and Glaucoma with binary gold standard vessel segmentation maps with ground truths [19].

  5. 5.

    E-Ophtha: Dataset is divided in two parts e-ophtha-MA and e-ophtha-EX(Exudates), total 381 MA and 82 exudates images are provided [19].

  6. 6.

    DIARETDB 0 and DIARETDB 1: Both the datasets consist of total 130 and 89 fundus images respectively. In which DIARETDB 0 have 20 normal & 110 DR images and DIARETDB 1 have 5 normal and 84 DR images. Both Datasets can be used for the task like exudates, MAs, HAs and NV detection [19].

  7. 7.

    MESSIDOR: Total 1200 images divided into 4 sets of 400 and again 400 images are divided into 4 sets of 100 each, provides DR and DME grading with MAs, HAs and micro-vascularization [19].

  8. 8.

    IDRiD: Total 516 fundus images with DR and DME grading as per the International clinical DR scale with hard exudates near macula respectively, also pixel level annotations are done 81 images by experts [20].

  9. 9.

    OCT Image Database (OCTID): It consists of total 500 images and grading is given in five categories such as normal 206 images, macular hole 102 images, Age related macular degeneration 55 images, central serous retinopathy, DR 107 images with corresponding ground truths [18].

  10. 10.

    Retinal Fundus Multi-Disease Image Dataset (RFMiD): This is the most recently developed and publicly available retinal fundus images dataset for multiple eye disease detection. It consists of 3600 retinal fundus images which are captured using three different cameras with resolution (2427 images) 2144 × 1424, (467 images) 4288 × 2848, (306 images) 2048 × 1536 respectively [21].

  11. 11.

    KMCM, India: Ophthalmology department of Kasturba Medical College, Manipal, India provided the dataset of 340 fundus images for the study conducted by Ganesan et al. [22].

  12. 12.

    Retina Identification Database (RIDB): This dataset focuses on retinal identification and recognition. This dataset consists of 100 images of retina. Dataset plays a significant role in retinal feature identification [23].

Literature Review on Automated Identification of DR

DR is the significant condition of disease found in diabetic patients which causes complete vision loss and if neglected or not diagnosed and not treated at early stages shows clinical features or abnormalities like microaneurysms, hemorrhages and neovascularization are seen in patients of DR [24].Hence automated analysis of retinal fundus images is the solution to help ophthalmologist to perform eye disease screening where retinal image classification, segmentation, feature extraction, pattern recognition, advance imaging and processing techniques, machine learning and deep learning algorithms, neural networks etc. are widely used [11].This section of the paper gives an overview on the methods and algorithms used in automatic detection or segmentation of OD, Blood vessels, lesions detection, DR,DME detection.

Optic Disc (OD) Segmentation

It is being observed that in the detection of DR, system must detect anatomical structures such as Fovea, OD and abnormalities. Hence most the researchers have focused on multiple segmentation methods to segment or locate them [12]. OD more glowing part of retina than other surrounded area generally has round or oval shape and macula is found close by the center of retina and transient to OD [2]. OD is considered to be brighter than other anatomical structures, hence algorithms based of intensity variations are used for OD detection but they fail when distracted by bright artifacts, CWS, exudates [25]. This section covers methods for segmentation of Optic Disc.

Aquino et al. [26] have proposed new template based method which applies circular Hough transform after morphological and edge detection technique and this method applied on MESSIDOR dataset and achieved 99% accuracy.

Kamble Ravi et al. [27] introduced a new approach for spotting OD and Fovea in which after preprocessing of image, one dimensional intensity analysis is performed where signal peak valley detection is used to locate OD in time domain and signal valley analysis is used to locate fovea in frequency domain. Framework evaluated on MESSIDOR dataset and achieved 99.75% accuracy for OD detection and 99.66% accuracy for fovea detection.

Zou Beiji et al. [28] introduces a new framework on verification model which integrates two sets strategies constructed on intensity and vascular data for detecting OD location. Verification model focuses on bright candidate regions, remove inaccurate area and classification. Introduced framework experimented on STARE, DRIVE, DIARETDB0, DIARETDB1 datasets and achieved 96.3%, 100%, 100% accuracy respectively.

Bin Gui et al. [29] proposed OD detection method based on improved corner detection approach. Simplified fast corner detection and improved Harris corner detection algorithms followed by preprocessing of image. Corner point is based on the property that distribution of corner points are dense at OD and using this property detection window detects maximum number of corner points which indicates OD position. This approach achieved 100%, 86.5% and 99.25% accuracy on DRIVE, STARE and MESSIDOR datasets respectively.

Baisheng Dai et al. [30] have described a novel method designed on variation model with multiple energies. SIFT and HOG feature extractors are used to extract shape, color and texture feature from image patch which are used for training purpose. Blood vessels present in ROI are removed using Morphological closing and opening. OD shape is obtained through principal component analysis and at the beginning of the process to determine OD shape Circular Hough Transform is applied. Proposed algorithm is tested on MESSIDOR, ONHSD and DRIONS datasets and achieved 99.17%, 100% and 100% accuracy respectively.

Niharika Thakur et al. [31] researchers have introduced a composite approach for OD and OD cup segmentation that is Level set based Adaptively Regularized Kernel Based Intuitionistic Fuzzy C means (LARKIFCM). Image cluster is obtained using Adaptively Regularized Kernel based Intuitionistic Fuzzy C means (ARKIFCM). OD and OD cup are effectively segmented in proposed method if positive features of combined approaches are taken. Accuracy achieved is 93.9%, 92.99% and 95.01% on Rim-One, Drishti and MESSIDOR datasets respectively (Table 2).

Table 2 Retinal Images Dataset table for study of DR, DME and Retinal features
Full size table

Rehman, Zaka Ur et al. [33] have presented multi-parametric OD localization technique using region based statistical and textural features. Simple linear iterative technique is applied after preprocessing of image after which highly discriminative features are extracted and passed to SVM, RF or AdaBoost classifier.

Table 3 shown below covers few more studies done for Optic Disc and Optic Cup segmentation.

Table 3 Methodology used for segmentation of OD
Full size table

Blood Vessel (BV) Segmentation

As shown in Table 1 main clinical feature seen in the PDR stage is nothing but neovascularization so it is very essential to accurately segment blood vessels as these newly generated blood vessels don’t support regular functionalities done by normal retinal blood vessels and differ in terms of length, diameter and path than regular blood vessels. This section focuses on the techniques for automatic BV segmentation.

Pachade Samiksha et al. [41] introduced a novel unsupervised technique which utilizes adaptive filter and mathematical morphology based operation to segment BV. Preprocessing technique is applied to remove noise. Adaptive manifold filter is used to enhance the image and contrast. Limiting adaptive histogram equalization is applied on enhanced image obtained in second step to get more enhanced image and at last simple thresholding method is applied to obtain segmented vessel. Method achieved 95.13% accuracy on DRIVE and 95.53 on STARE dataset.

Pachade Samiksha, et al. [42] developed a framework to detect blood vessel and width measurement in color fundus images and SLO images. To enhance the vessels linear recursive filtering is applied and Morphological operations, background estimation, and iterative thresholding are combined together for BVs segmentation. Vessel centerlines extracted using graph based algorithm and image profiles are used for localizing vessel edges. Accuracy achieved on DRIVE–95.52%, STARE–95.43%, IOSTAR–95.61% and RCSLO–96.05%.

Fan et al. [43] presented a new method for BV segmentation using region features of vessels. Isotropic undecimated wavelet transform and morphological reconstruction are used to obtain two binary images. Major vessel is obtained through common region of two images. To identify the vessel containing region at the beginning vessel images are separated based on features. Accuracy obtained on DRIVE and STARE datasets is 95.8%.

Jothi  et al. [44] have described a new algorithm based on Frangi filter based on the concept that vessels in image consists of a small area to be considered and are lighter or darker than their background.

Samuel et al. [45] introduced CNN based method Vessel Specific Skip Chain Convolution Network for BV segmentation which is used to detect vessels in fundus as well as X-ray Coronary Angiogram images.

Table 4 shown below covers few more studies done for Blood Vessel Segmentation.

Table 4 Methodology used for segmentation of Blood vessels
Full size table

Lesion Detection

DR grading is done on the basic of clinical features which are mentioned in Table 1 and shown in Fig. 3 such as MAs and HAs considered as red lesions, CWS considered as white lesions, exudates are yellowish or white object. This section focuses on the methods used for detection of MAs, HAs, CWSs, Exudates lesions.

Mane et al. [52] introduced a strategy for location of MAs and HAs and uses adaptive approach to matched filtering for estimation and extraction. SVM classifier is used for feature extraction. Method is tested on DIARETDB1 with 96.62% accuracy.

Du Jingyu et al. [53] developed a method where to differentiate between MAs and other vascular structures cross-sectional changes in MA profile are considered for which local cross section transformation method proposed by author and multi-feature combinations are used to detect MAs. RusBoost classifier is used for classification. Local minimum region extraction block filter is used for MA extraction.

Manjaramkar Arati et al. [54] presented a connected component method based on Maximally Stable External Regions (MSER) for HAs detection. MSER algorithm is used for extracting co-variant regions and feature extraction by region filtering. Sensitivity and specificity at image and lesion level is tested on DIARETDB1 and MESSIDOR datasets as 96.45%, 97.64% and 94.89%, 98.9% respectively.

He, Yunlong, et al. [55] described a new approach based on multispectral retinal imaging for DR lesions spotting. Visual descriptor Local binary pattern, generalized low rank approximation of matrices, supervised regularization term with Gaussian kernel based SVM are used for detection of MAs,HAs,CWSs and Macular Edema and have achieved 98.1% accuracy.

Khojasteh, Parham, et al. [56] have presented a comparative analysis on multiple supervised and unsupervised classifiers in DL methods. Also proposed a method which uses ResNet-50 with SVM to detect exudates and have obtained better results when tested on e-Ophtha dataset with 99% accuracy.

Munuera‐Gifre, Eduardo, et al. [57] performed an analysis to find out the dispersion of Type-2 DM retinal lesions to check if they are comparative between pair of eyes or arbitrary or they follow some design or pattern. Analysis results obtained proved that lesion distribution follows a particular pattern and which can be further used for DR detection CAD systems.

Table 5 shown below covers few more studies done for lesion detection.

Table 5 Methodology used for abnormalities or lesion detection such as Microaneurysms (MAs), Exudates(EX), Hemorrhages (HAs)
Full size table

DR Detection

In past few years tremendous of research work is done on computerized detection of DR and DME. Section covers on strategies and algorithms for detection of DR and DME.

Shirbahadurkar et al. [65] have implemented a modified tele-ophthalmology system for DR classification which uses holoentroy enabled decision tress classifier. Performance of method is evaluated using tenfold cross validation which achieved 98.88% accuracy.

ElTanboly, Ahmed et al. [24] have developed CAD system for DR grading for OCT images. It follows three steps, localize and segment OCT images using Joint Markov Gibbs Random Field (MGRF) model of intensities, derive the features of segmented OCT image, find out important features and classify them using Multistage deep fusion classification network (DFCN). System gave 92% accuracy when tested on private dataset considered by researchers.

Li Xiaomeng et al. [66] presents a new method for DR and DME grading as cross disease attention network (CANET). This method mainly focuses on the detection of useful features of individual disease with disease attention module. Disease dependent attention module is used for checking internal relationship between diseases. Training of the data done using ResNet. Algorithm is evaluated on on IDRiD and MESSIDOR datasets.

Hagos et al. [67] have developed a smart phone based approach for detection of DR. Inception based CNN and binary decision tree based ensemble of classifiers with android based smartphone application software are proposed and used in DR detection. Results are compared with state of art method with minimum accuracy achieved is 87.12% and maximum accuracy with 99.86%. Objective of this method is to come up with handy technology to diagnosis DR at any point of time with fastest way and at any remote location.

Sisodia, Dilip Singh et al. [68] introduces a method for early detection of DR using preprocessing and feature extraction. In preprocessing of image green channel of retinal image is considered, for contrast histogram equalization and image enhancement and resizing is done. Lesion detection is done using extraction of fourteen features. Removal of OD is done by high contrast circular shapes and fray scale closing operator respectively. SVM classifier is used for classification.

Table 6 shown below covers few more studies done for DR and DME detection.

Table 6 Methodology used for DR and DME detection
Full size table

Future Scope and Challenges

In literature survey it is observed that Diabetic retinopathy and Diabetic Macular Edema patients are instructed to do regular check so that they don’t suffer from complete vision loss as these two are silent eye threatening disease. It is also observed that new methodologies considered for CAD are surely better and quicker in giving results than traditional methods but there is still need of developing a bigger dataset from all around the world considering regional factors and lifestyle of people. Most of the studies are focusing on fundus imaging techniques and researchers have developed multiple novel methodologies using CNN, image processing techniques or deep learning or hybrid models which are also giving very good accuracy results. With all this it is also observed that Optical Coherence Tomography images should be used widely for detection of DR and DME as this imaging technique has special importance in terms of understanding the severity patterns layer by layer of retina also helps in understanding retinal blood vessel width changes and neovascularization which have not received much focus by researchers. Most of the studies have got maximum success rate when focused on particular type of abnormality or lesion in DR or DME detection or have not considered all class grading of disease which is indirectly compromising the effectiveness of research. The main challenge observed in lesion detection is how the image is captured or in which conditions as brightness or pixel intensity values can lead to confusion while detecting normal retinal features, hence strong color model algorithm should be introduced.

Conclusion

The review paper covers maximum recent two–three year papers, have also covered the information about diabetes mellitus and its side effects on eyes and provides detailed study about worldwide statistics of people suffering from diabetes. The next topic covered in paper is about vision threating eye disease DR and DME with their clinical background information with differentiating importance of CAD method over traditional eye checkup method. We have also covered publicly and privately available retinal dataset used for detection of DR and DME. In the next section of paper more focus is given on discussion of existing methodologies used for OD detection, blood vessel detection, lesion detection and disease detection with their performance measures for each used dataset. In the last section observations by authors with challenges and future scope are covered. CAD system will be very helpful in diagnosis of DR and DME and this will not replace the doctors but will be helping them in fast and time saving diagnosis.