Introduction

Diabetes mellitus (DM) is commonly known as diabetes. It is a group of metabolic disorders which are characterized by the high blood sugar [1,2,3]. Diabetes can lead to many serious long-term complicated disease like cardiovascular disease, stroke, kidney failure, heart attack, peripheral arterial disease, blood vessels, and nerves [4, 5]. About 122 million people were affected by diabetes in worldwide in 1980 and this figure was reached about 422 million in 2014 [6]. The figure will be reached about 642 million in 2040 [7]. Moreover, there were directly about 1.6 million deaths due to diabetes [8]. Therefore; it is an alarming figure to us. The number of diabetic patients is increased day by day as a result deaths are also increased day by day. Diabetes can be divided into three types as (i) type I diabetes (T1D), (ii) type II diabetes (T2D), and (iii) gestational diabetes (GD) [9]. T1D are normally in young adults whose age is less than 30 years. The symptoms of T1D are polyuria, thirst, constant hunger, weight loss, vision changes and fatigue [10]. T2D occurs in adults over 45 years which are often associated with obesity, hypertension, dyslipidemia, arteriosclerosis, and other diseases [11]. The third type of diabetes is gestational diabetes. Actually pregnant women are affected by gestational diabetes.

The analysis of diabetes data is a challenging issue because most of the medical data are nonlinear, non-normal, correlation structured, and complex in nature [12]. The ML-based systems have dominated in the field of medical healthcare [12,13,14,15,16,17,18,19,20,21] and medical imaging such as stroke, coronary artery disease, and cancer [22,23,24,25,26]. Moreover, ML-based systems can be used as both feature selection techniques (FST) and classifiers. It also helps the people to accurately diagnosis of diabetes and the best classifier is the most important problems for accurate diabetes risk stratification. There were various ML-based systems used to classify and predict of diabetic disease like linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), naïve Bayes (NB), support vector machine (SVM), artificial neural network (ANN), feed-forward neural network (FFNN), Adaboost (AB), decision tree (DT), J48, random forest (RF), Gaussian process classification (GPC), logistic regression (LR), and k-nearest neighborhood (KNN) and so on [12, 13, 17,18,19,20,21,22,23,24,25,26,27,28,29,30]. The overview of the proposed ML-based system is shown in Fig. 1. We hypothesize that the combination of logistic regression (LR) based FST along with the training-based four classifiers can be accurately diabetes risk stratification.

Fig. 1
figure 1

Overview of the proposed ML-based system

Full size image

Thus, in this study we have adopted LR model to identify the risk factors of diabetes disease based on p-value and odds ratio (OR). We also adopted four applicable and important ML-based classifiers as: NB, DT, AB, and RF. The main objective of this study is to identify the most significant factors of diabetes disease based on LR model and develop a ML-based system for the accurate risk stratification of diabetes disease. Finally, we thus summarize the main contributions in this study as follows:

  • To identify the risk factors of diabetes disease using LR model based p-values and OR.

  • To choose the best ML-based system by selecting the best protocol (three partition protocols: K2, K5, and K10) and classifier (four classifiers: NB, DT, AB, and RF) combination based on accuracy (ACC), sensitivity (SE), positive predictive value (PPV), negative predictive value (NPV), F-measure (FM), and area under the curve (AUC).

  • To validate our proposed ML-based system, we demonstrate the same performance using Indian liver patient dataset.

The paper is organized as follows: “Materials and methods” section represents materials and methods, including description of the dataset, statistical analysis, machine learning system, feature selection techniques, data partitioning, prediction model, and performance evaluations of the classifiers. The results are discussed in “Results” section. “Discussion” section represents the discussions in detail along with key difference between our proposed ML-based system and previous work, strength and extension of the study, and summary of the current study. Finally conclusion is presented in “Conclusion” section.

Materials and methods

Data

The diabetes 2009–2012 dataset have been used in this study, derived from the National Health and Nutrition Examination Survey (NHANES). NHANES is an ongoing and cross-sectional study and population sample survey of United States (US) population. The dataset consisted of 9858 respondents. Respondents were identified as a diabetic patient if they met with at least one of the following criteria: plasma fasting glucose ≥ 126 mg/dL, serum glucose ≥ 200 mg/dL, glycohemoglobin ≥ 6.5%. There were about 760 diabetic respondents and 9098 control respondents. There were some missing values and unusual observations in the dataset. Excluding the missing values and unusual observations from the dataset, there were a total of 6561 respondents with 657 diabetic and 5904 controls. Detailed description of the dataset is shown in Table 9 in Appendix 1. The dataset is public domain survey and freely available in online.

Statistical analysis

The baseline characteristics of the study population are presented as mean ± SD (standard deviation) for continuous variables and number (percentage) for the categorical variables, respectively. Differences in variables between diabetic patients and control are analyzed by independent t-test for continuous variables and Chi square test for categorical variables. All of the tests are two-tailed and considered as significant factors whose p-values are less than 0.05. The demographic and clinical characteristics of the diabetic patients are described in Table 1. Data are analyzed using Stata-version 14.10 and R-i386 3.6.1.

Table 1 Demographic and clinical characteristics of diabetic patients
Full size table

Machine learning system

The main objective of the ML-based system is to classify and predict of the diabetes disease. The overview of the proposed ML-based systems has been shown in Fig. 1. The training/test set paradigm of the entire ML-based systems has been shown in Fig. 2. The first step is to divide the dataset into two sets such as training set and test set. The training and test sets are separated by dotted line as online and offline classification system. In the second step, the most significant risk factors of diabetes disease using LR model based on p-value and OR. The next stage is to adapt three partition protocols (K2, K5, and K10). And each protocol is repeated into 20 trials (T). We have also adopted four classifiers as: NB, DT, AB, and RF, respectively. The fourth step is to estimate the training classifier coefficients, and then the test classifiers have been applied to classify the patients into two categories as diabetic vs. control. Finally the performances of the classifiers are evaluated using six performance parameters, namely: ACC, SE, PPV, NPV, FM, and AUC.

Fig. 2
figure 2

The training/test set paradigm of the ML-based system

Full size image

Feature selection

In machine learning and statistics, feature selection is known as variable selection. It is mainly exercised for choosing the relevant factors to use in ML-based system. There are five reasons for selecting the best subset of predictors as: (i) run the ML-based system easily and interpret the results, (ii) avoid curse of dimensionality, (iii) save computational cost as well as time, (iv) reduce the over-fitting, and (v) improve the classification accuracy. There were some commonly used FST in ML/statistics namely: RF [13, 31, 32], LR [33, 34], mutual information (MI) [13, 35], principal component analysis (PCA) [13, 35, 36], analysis of variance (ANOVA) [13, 37, 38], and Fisher’s discriminant ratio (FDR) [13, 35, 39]. In this study, we have used LR model to identify the risk factor for diabetic disease based on p-value (p < 0.05) and OR.

Logistic regression

Logistic regression (LR) is a supervised learning while the predictors are continuous/discrete and response variable is dichotomous (diabetic vs. control). LR model is used to estimate the probability of a binary response based on one or more predictors. LR also measures the relationship between response and one or more predictors by estimating probability logit function. The logit of response variable (Y) is the linear combination of the predictors (X) which can be written as follows:

$${\text{logit}}({\text{P}_\text{j}} ) = \log_{\text{e}} \left( {\frac{{{\text{P}_\text{j}} }}{{1 - {\text{P}_\text{j}}}}} \right) = \mathop{\sum}\limits_{{{\text{i}} = 0}}^{\text{K}} {\beta_{\text{i}}} {\text{X}_\text{i}}$$
(1)

where \({\text{P}}_{\text{j}}\) is defined as the probability for Y = 1 (diabetic) and 1 − Pj (control) is defined when Y = 0. βi’s \(\left( {{\text{i}} = 0,1, \ldots ,{\text{K}}} \right)\) are the unknown regression coefficient, K is the total number of predictors (14 factors) and Xi’s \(\left( {{\text{i}} = 1, \ldots ,K} \right)\) are the predictors and \({\text{X}}_{0} = 1\). We estimate the regression coefficients by maximum likelihood estimator (MLE) and get easily OR by the exponent of the regression coefficients. We can easily test the regression coefficients/ORs by applying z-test and select the features corresponding to the regression coefficients/ORs whose p-values are less than 0.05.

Data partitioning

Data partitioning is known as cross-validation (CV) protocol. It is mainly used for dividing the given dataset into two subsets as: (i) training set and (ii) validation/test set. There are lots of CV protocols, used for partitioning the dataset to reduce the variability. The tenfold CV protocol is commonly used in both ML and statistics, whereas the dataset is divided into ten equal parts while the nine parts are used as a training set for ML-based system and remaining part is used as a validation/test set. This protocol is repeated into 20 times while run the ML-based system for test set and calculate the classification accuracy at each protocol and then take the average. This is termed in the manuscript as K10 where 10 indicates the number of total partition during ML-based system. Similarly, the well-known data partitioning protocols are K2, K4, and K5, respectively, depending on the percentage (%) of the training set as 50%, 75%, and 80%, while rest of the parts are treated as a test set. In this study, we have used three partition protocols as K2, K5, and K10, respectively.

Prediction model

In this study, we have used four sets of ML-based classifiers as NB, DT, AB, and RF. The brief discussions of these classifiers are discussed as follows.

Naïve Bayes

Naïve Bayes (NB) is a simple probabilistic classifier based on Bayes theorem. The main assumption of NB is features are mutually independent [40]. In the recent study, it is used to diagnosis of different types of disease especially diabetes disease [12, 13]. The NB classifies the data using Bayes theorem as follows:

$${\text{P}}({\text{z}}|{\text{w}}_{1} , \ldots ,{\text{w}}_{\text{n}} ) = \frac{{{\text{P}}\left( {\text{z}} \right) \pi_{{{\text{i}} = 1}}^{\text{n}} P({\text{w}}_{\text{i}} |{\text{z}})}}{{\pi_{{{\text{i}} = 1}}^{\text{n}} {\text{P}}\left( {{\text{w}}_{\text{i}} } \right)}}$$
(2)

where, z is the response variable, w1, w2,…, wn are input variables; P(z|w1, …, wn) is the conditional probability distribution of z given w1, w2,…, wn; P(z) is the marginal probability distribution of z; \({\text{P}}({\text{w}}_{\text{i}} |{\text{z}})\) is the conditional probability distribution of \({\text{w}}_{\text{i}}\) given z; p (wi) is the marginal probability distribution of wi; π is the product symbol. We found the probability of z given the set of inputs and picked up the output with maximum probability. The corresponding classifier, a Bayes classifier, is the function that assigns a class label as follows:

$${\text{z}} = {\text{argmax}}_{\text{z}} {\text{P}}\left( {\text{z}} \right) \pi_{{{\text{i}} = 1}}^{\text{n}} {\text{P}}({\text{w}}_{\text{i}} |{\text{z}})$$
(3)

Decision tree

Decision tree (DT) is a supervised learning that can be used as regression tree while the response variable is continuous and as classification tree while the response variable is categorical. Whereas the input variables are any types as like graph, text, discrete, continuous, and so on in the case of both regression and classification. A decision tree is a tree structure based model which describes the classification process based on input features [41]. The steps of DT as follows: (i) construct a tree with its nodes as input features; (ii) select the feature to predict the output from the input features whose gives the highest information gain; (iii) repeat the above steps to form sub trees based on features which was not used in the above nodes [30].

Adaboost

Adaboost (AB) means adaptive boosting, a ML-based technique. Freund and Schapire introduced AB algorithm in 1996 [42] and won Gödel prize in 2003. It is used in conjunction with different types of algorithm to improve classifier’s performance. AB is very sensitive to noisy data and outliers. In some problems, it is less susceptible to the over-fitting problem than other learning algorithms. Every learning algorithm tends to suit some problem types better than others, and typically has many different parameters and configurations to adjust before it achieves optimal performance on a dataset. AB is known as the best out-of-the-box classifier [34].

Random forest

Random forest (RF) is a ML-based classifier by constructing decision trees. This algorithm was first proposed by Breiman [43]. RF can be also used as both regression and classification. RF can be used in several biomedicine research [20, 44], especially diagnosis of diabetes [12, 13]. The steps of RF as follows:

  • Step 1: divide the dataset into two parts as training set as well as test set. From the given training set, create a new dataset using the bootstrapping method.

  • Step 2: construct a DT based on the results of step 1.

  • Step 3: repeat step1 and step2 and produce many trees which consist of a forest.

  • Step 4: use every tree in the forest to vote for given input variables.

  • Step 5: compute the mean votes for each class. The class which gives the highest votes that are belongs to the classification label for the given input variables.

  • Finally compute the classification accuracy of RF-based classifier.

Performance evaluations

Many statistical parameters may be used to compare the performance of the classifiers. In this study, we have used ACC, SE, PPV, NPV, and FM. ACC is the proportion of the sum of the true positive and true negative against total number of population. SE is the proportion of the positive condition against the predicted condition is positive. PPV is the proportion of the predicted positive condition against the true condition is positive. NPV is the proportion of the predicted negative condition against the true condition is negative. FM is defined as the harmonic mean of the precision and recall.

Results

Demographics and clinical characteristics of diabetic patients

The patient’s demographics and clinical characteristics are shown in Table 1. A total of 657 (11%) from the pool of 6561 subjects are diabetic patients. In this study, we have taken 50% male and the overall age of the respondents are 47.18 ± 16.79 years. There are 55% male diabetic patients with average age 59.85 ± 13.22 years. It is observed that all attributes are highly statistically (p < 0.001) associated with diabetes.

Feature extraction using logistic regression

Table 2 shows that the effect of selected factors on diabetes using logistic regression. Its shows that age, education, BMI, systolic BP, diastolic BP, direct cholesterol, and total cholesterol are statistically significant factors for diabetes disease at 5% level of significance and the rest of the factors are insignificant. These seven factors are used for ML-based system to classify and predict of diabetes disease.

Table 2 Effect of selected factors on the diabetes using logistic regression
Full size table

Performance analysis of machine learning system

The comparison of the classification accuracy of four classifiers are shown in Fig. 3 for three partition protocols (K2, K5, and K10) while keeping the data size fixed (n = 6561). It also shows that the accuracy of all classifiers is increased as increasing the number of partition protocol from K2 to K5 to K10. It is observed that RF-based classifier performs better for all protocols compared to other classifiers. It is also observed that RF-based classifier gives the highest classification accuracy of 94.25% for K10 protocol whereas NB classifier gives the lowest classification accuracy of 86.70% for K10 protocol. The corresponding results are presented in Table 3. Moreover, the four performance evaluation parameters as SE, PPV, NPV, and FM of four classifiers for three partition protocols are shown in Table 4. Also the best performers of RF-based classifiers among four classifiers are validated using FM (see the last column of Table 4). We observe that RF-based classifier gives the highest FM of 96.88% for K10 protocol.

Fig. 3
figure 3

Comparison of accuracy of four classifiers for three partition protocols keeping data size fixed

Full size image
Table 3 Comparison of accuracy (%) of four classifiers for three partition protocols
Full size table
Table 4 Four performance evaluation parameters for 4 classifiers
Full size table

Figure 4 shows the effect of varying data size on the classification accuracy of four classifiers for three partition protocols. We divide the actual data size (n) into ten parts as follows: 656, 1312, 1968, 2624, 3281, 3937, 4593, 52,49, 5905, and 6561. Now, we take these training sets and calculate the classification accuracy of four classifiers for three partition protocols. It is observed that the net generalization yields the generalization cutoff of 70% of the cohort (patient pool of 4593 patients). That means we need at least 70% patients to achieve the generalization. It is observed that the classification accuracy is increased by increasing the data size (n). Figure 4 also shows the classification accuracy of all classifiers increases with increases in data size and RF-based classifier is performed better than others. In Appendix 2 shows the system accuracy of four classifiers varying of data sizes for K2 protocol (Table 10 in Appendix 2), K5 protocol (Table 11 in Appendix 2), and K10 protocol (Table 12 in Appendix 2). Then the system mean accuracy is calculated by averaging the classification accuracy of all classifiers over varying data sizes for three partition protocols (K2, K5, and K10). Table 5 shows the system mean accuracy of four classifiers for three protocols. It is also showed that RF-based classifier is performed better compared to others.

Fig. 4
figure 4

Effect of varying data size: Accuracy vs data size (n) for three partition protocols. a K2 protocol; b K5 protocol; and c K10 protocol. Medical arrows indicate the proposed method’s result

Full size image
Table 5 System mean accuracy (%) of four classifiers for three partition protocols
Full size table

Receiver operating characteristics (ROC) analysis

Receiver operating characteristics (ROC) is a graphical plot that is created by plotting sensitivity versus ‘1-specificity’. The area under the curve (AUC) which is computed from ROC curve is the indicator to evaluate the performance of the classifiers [45]. The value of the AUC lies between ‘0’ to ‘1’. R-i386 3.6.1 statistical software was used to compute the value of AUC from ROC curves. Figure 5 shows ROC curves of four classifiers for three partition protocols (K2, K5, and K10). It is observed that RF-based classifier is better for all partition protocols than NB, DT, and AB, respectively. The corresponding AUC values are presented in Table 6. It is also observed that the AUC of RF-based classifier for K10 protocol is 0.95 while NB, DT and AB are 0.82, 0.78, and 0.90, respectively.

Fig. 5
figure 5

ROC curves of four classifiers for three partition protocols: a K2 protocol; b K5 protocol; and c K10 protocol. Medical arrows indicate the proposed method results

Full size image
Table 6 Comparison of AUC of four classifiers for three partition protocols
Full size table

Validations of the proposed method

To validate our proposed method, we have used Indian liver patient’s dataset. The dataset has been taken from the University of California, Irvine (UCI) machine learning data repository [46]. The dataset consists of ten attributes and 583 patients. There are 416 liver patients and 167 non-liver patients. Our result denominates that RF based classifier gives the highest accuracy compared to others. Therefore, we may conclude that our proposed method is the best classifier for both diabetes dataset and Indian liver patient’s dataset. It indicates that our proposed method is validated (see Table 7).

Table 7 Validation of the proposed method using liver patient’s dataset
Full size table

Discussion

In this study, we have presented the ML-based system risk stratification to classify the patients into two classes as diabetic and control. Moreover, LR-based model has been adopted to determine the high risk factors of diabetes disease. The high risk factors have been selected based on p-values and odds ratio (OR). Moreover, four classifiers have been also adapted and compared their performance based on ACC, SE, PPV, NPV, FM, and AUC, respectively. Further, three partition protocols have been applied for generalization of classification and this process has been repeated for T = 20 times to reduce the variability. Also we have validated the ML-based system using Indian liver patient’s dataset. The next section represents the key comparison between proposed ML-based system and previous work.

Key difference between proposed ML-based system and previous work

Several papers in the literature are focused on the identification of high risk factors as well as sophisticated classification the diabetes disease. Zou et al. [47] studied on the diagnosis of diabetes dataset. The dataset was taken from the hospital physical examination data in Luzhou, China. The dataset contained 14 attributes and consisted of 220,680 patients. Among them, 151,598 (69%) patients were diabetic and 69,082 (31%) were control. They applied PCA and minimum redundancy maximum relevance (mRMR) to reduce the dimensionality and also K5 cross-validation protocol adopted to examine the data. They also applied three classifiers as: DT, NN, and RF to classify the diabetes patients and demonstrated that RF based classifier gave the highest classification accuracy of 80.84%. Maniruzzaman et al. [12] applied LDA, QDA, NB, and Gaussian process classification (GPC) on the Pima Indian diabetes (PID) dataset to classify the diabetic patients. They adopted two partition protocols as K5, and K10. They showed that GP-based classifier with radial basis kernel (RBF) gave the highest classification accuracy approximately 82.00%. The same authors firstly identified the outliers in the PID dataset using inter-quartile range (IQR). If the outliers were detected, then they were replaced the outliers by the median. They also replaced the missing value by median. They adopted six feature selection techniques as PCA, LR, MI, ANOVA, FDR, and RF to extract the features. They also adopted ten classifiers as: LDA, QDA, NB, NN, GPC, SVM, AB, LR, DT, and RF to classify the diabetes patients. They found that the combination of RF based feature selection technique and RF based classifier gave the highest classification accuracy of 92.26% compared to others [13].

Ahuja et al. [30] used PID dataset in his study. The dataset consisted of 768 patients and 10 attributes. The dataset had some missing values and they were replaced the missing values by median. LDA was used to extract the feature selection. They applied five classification algorithms as: SVM, multi-layer perceptron (MLP), LR, RF, and DT. They showed that LDA with MLP based classifier gave the highest classification accuracy of 78.70%. Sisodia et al. [29] did not apply any feature selection techniques to extract the feature. They performed K10 protocol and applied SVM, NB, DT classifiers and got the highest accuracy 76.30% of NB classifier compared to others. Yu et al. [48] took diabetes dataset from the 1999–2004 US NHANES to develop a SVM model to classify diabetes patients. The dataset consisted of 6214 patients (1461 diabetic patients and 4853 patients were control). Firstly, they optimized the kernel and chose the best kernel for SVM based on the classification accuracy. They adopted K10 cross-validation protocol and four types of kernel as: linear, polynomial, sigmoid, and RBF respectively. They got that SVM with RBF kernel gave the highest classification accuracy of 83.50%. Semerdjian et al. [49] used 5515 total samples were available from the NHANES 1999 to 2004 dataset. They identified the highest risk factors based on RF. Five set of classifiers (LR, KNN, RF, Gradient boosting (GB), and RF) adopted to predict the diabetes status based on the 16 attributes and the performance of GB based classifier was higher (AUC: 0.84) compared to others. Mohapatra et al. [50] applied MLP and found that MLP gave the classification accuracy of 77.50%. Pei et al. [51] also applied DT and gave 94.20% classification accuracy. However, in this study; we have selected LR based feature selection technique to identify the risk factors of diabetes disease. The results show that the combination of LR and RF-based classifier gives 94.25% ACC and 0.95 AUC are the highest as compared to conventional techniques (see Table 8).

Table 8 Key difference between proposed study and previous studies in literature
Full size table

Strength and extension of the study

This paper shows the risk stratification and classification of diabetes patients while there are 6561 respondents with 657 diabetics and 5904 controls. Our result demonstrates that the overall accuracy of ML-based system is 90.62%. Moreover, the combination of LR based feature selection technique and RF-based classifier gives the highest classification accuracy of 94.25% and 0.95 AUC for K10 protocol. Nevertheless, the presented system can still be improved. Further, preprocessing techniques may be used to replace missing values by various missing value imputation techniques like: mean or median, expectation maximization (EM) algorithm, K-nearest neighbors (KNN), fuzzy K-means (FKM), and singular value decomposition (SVD). Moreover, there are various techniques of feature extraction, feature selection (PCA, different statistical tests, FDR, RF, etc.), and classifiers, namely: NN, GPC, SVM, deep learning (DL) and so on.

Summary of the current study

In this section, we summarize the current study at a glance as follows:

  1. 1.

    Data:

    1. (a)

      Extraction: Extract the dataset from the National Health and Nutrition Examination Survey (NHANES) into a dta (Stata) format.

    2. (b)

      Data cleaning: Drop the missing values and outliers from the dataset.

    3. (c)

      Feature extraction: Identify the high risk factors using LR for prediction.

  2. 2.

    Modeling:

    1. (a)

      Model selection: The popular predictive models were selected for this project:

      (i) Naïve Bayes; (ii) Decision tree; (iii) Adaboost; and (iv) Random forest

    2. (b)

      Data split: Discussed in the “data partitioning” section.

  3. 3.

    Evaluation:

    1. (a)

      Metrics: Reporting of accuracy, sensitivity, positive predictive value, negative predictive value, F-measure, and area under the curve to evaluate the classifiers.

    2. (b)

      Interpretation: examining results of metrics to compare the classifiers and finally conclude the experiments.

Conclusion

Diabetes mellitus (DM) is commonly known as diabetes. It is a group of metabolic orders which are characterized by the high blood sugar. Our hypothesis was used in ML based system using LR-RF combination for feature selection technique and classifier gave the highest classification accuracy. Our results demonstrated that our proposed combination reached an accuracy of 94.25% for K10 protocol. Moreover, a comparative analysis was conducted using four classifiers, one feature selection technique, and three partition protocol (total 12) experiments. It would be interesting in future to see classification of other kinds of medical data to be adapted in such a framework creating a cost-effective and time-saving option for both diabetic patients and doctors.