Keywords

1 Introduction

1.1 Multiclass Classification

Classification is one of the crucial tasks in the field of machine and deep learning. In this era where there are data available is in millions carries with them some hidden information or patterns that are needed to be recognized for intelligent decision making. The two important forms of data analysis are classification and prediction which are being used to build a model for describing important class labels or to predict future data trends [1]. In this paper these two important concept classification and prediction were used. Classification used was multiclass classification which classify whether the given medical imaging is brain, chest, or knee and then prediction concept was used to predict whether the organ identified had disease or is normal.

Multiclass classification refers to the problem of classifying instances into more than two classes. In this problem encountered was to identify the disease of the patient given medical imaging in the form of MRI, X-ray, endoscopy, ultrasound, etc. There are three ways to solve multiclass classification, first is to extend binary classifier, second is to break the multiclass classification problem into several binary classification problem and third is Hierarchical classification methods [1].

Extending the binary classifier to solve multiclass classification problem is known as extensible algorithms. Some of the algorithms that can be naturally extends are neural networks, decision trees, k-Nearest Neighbor, Naive Bayes, and Support Vector Machines.

The method of breaking the multiclass classification problem into several binary classification problem is known as decomposing into binary classification. There are several methods for doing so which include OVA (one versus all), AVA (All versus all), ECOC (Error correcting output coding) and Generalized coding [2]. In OVA approach, the problem of classifying among k classes is reduced to k binary problems, where each problem is distinguishes a given class from the remaining k-1 classes. In the n = k classifier is required where every classifier is trained with positive examples belonging to class k and negative examples belonging to other classes k-1. In OVO each class is being compared with every other class and a classifier is built to discriminate between each class discriminating all other and voting is performed among the classifier, and whichever gets the maximum votes wins. In ECOC, the idea is to train n binary classifier to distinguish among k classes and every class is given a codeword of length n according to binary matrix m where m denotes a unique number representing a unique class. Generalized coding is the extension of ECOC where coding matrix can take values − 1, 0, and +1.

Hierarchical classification solves the problem of multiclass classification problem by diving the output space i.e., into the tree. The nodes are divided into multiple child nodes till each child nodes represents only one class.

In this paper, CNN was used to perform the multiclass classification problem and the detail regarding the classification will be discussed further in the paper.

1.2 Coronavirus

The Coronavirus disease 2019 (COVID-19) that has been originated from China has now been most frightening and death threatening disease that has spread all over the world. It not only affected the health of the public, but all affected the economy of the countries. It is a respiratory disease caused by SARS-COV-2 virus that seems like viral pneumonia but can result in death if not treated immediately [3]. According to WHO (World Health Organization), COVID-19 has affected over 280,119,931 people all over the world including 5,403,662 deaths as of 28th December 2021. It is challenging the medical system and needs to be taken care of as if diagnosed earlier people may be able to save by the treatment provided. Presently there are three ways for diagnosis of the COVID-19 RT-PCR, CT-scan and CXR (chest Radiography) [4,5,6,7]. In this paper, CXR was used to perform the diagnosis of the COVID-19 as it provides reliable results [4]. SVM algorithm was to determine whether the CXR image is affected with COVID-19, or it is normal.

1.3 Brain Tumor

Brain tumor is formed by the excessive production and proliferation of the cells in the skull. Since the brain is the body’s command center, tumors can impose strain on the head and can have severe impact on the human health [8]. As the people dying due to brain tumors are rising day by day hence the early diagnosis is important. There is several medical imaging that can be useful for the diagnosis available presently namely CT (Computed Tomography), SPECT (Single-Photon Emission Computed Tomography), PET (Positron Emission Tomography), MRS (Magnetic Resonance Spectroscopy) and MRI (Medical Resonance Imaging) [9]. In this paper for the diagnosis of the Brain tumors MRI images were used, where CNN was used for identifying whether it is the image of brain MRI and SVM for predicting whether it has brain tumor or is normal.

1.4 Knee Osteoarthritis

A Knee osteoarthritis (OS) is a serious disease that is the world’s fourth greatest cause of disability. It is commonly known as degenerative joint disease and is caused by fatigue failure as well as the gradual loss of articular cartilage. It is a disease that usually worsens with time and can lead to impairment. The severity of clinical symptoms varies from person to person [10]. Despite this, risk analysis models for predicting pain progression in individuals with knee OA are still scarce. Current risk analysis models generally rely on demographic, clinical, and radiographic risk variables, or thorough MRI examination analysis. As a result, new and improved methodologies are required to develop ubiquitous, cost-effective, and easily obtained risk assessment models for forecasting pain development in knee OA patients [11]. In this paper X-Ray of the knee was used for the diagnosis and the model was build using CNN and SVM to predict whether a person is suffering from Knee OS or not.

2 Literature Survey

2.1 Deep Learning Based Medical Image Multiclass Classification

In [4] authors propose a DGCAN-based CNN model that makes use of CRX image for the classification of the images into the three categories which were normal, pneumonia and COVID-19 and confirmed that the performance of their proposed model is much better that the existing model as the proposed model makes use of DGCAN which is a multiple neural network that make use of random noise of the images and first extract the local features on several levels and then focuses on extracting the global images. Aishwarya et al. [3] in their paper does a review on the detection of COVID-19 using various dataset. The focus of their work includes studying detection using deep learning and machine learning technique on various datasets available. In [8] authors propose a web- based software that classify the brain tumors into three categories glioma, meningioma and pituitary using the CNN model and produces accuracy of more than 98%. Francisco et al. [9] proposes a CNN model for brain tumor into same categories as [8] that make use of multi-scale approach in which input images are processed in three different spatial scales and produces and accuracy of 97% which according to authors was higher than the existing models with the same datasets. G.wan. et al. [11] develops a deep based risk assessment model and compares with the traditional methods and concluded that deep learning model produces better diagnostic performances using baseline knee radiographs.

2.2 SVM Classifier for Image Classification

In [12] their suggested research used deep learning approach for the classification of the sugarcane as diseased or not, authors make used of three different feature extraction technique and output is given as input to seven different machine earning classifier and based on the result using RC and AUC used the VGG-16 as feature extractor and SVM as the classifier as it produces maximum AUC of 90.2%. Reem Alrais [13] proposes SVM model for classifying the medical image into two types of tumors. Author before classifying the images uses DWT (Discrete Wavelet Transform) to remove noise from the images and PCA (Principal Component Analysis) for reducing the dimensional feature to get the only required storage and computational space. In [14] authors presented a detailed study on the application of the SVM in image classification and how the enhancement of the image may lead to better accuracy in the classifier. Renukadevi et al. [15] in their study concluded that SVM accuracy changes with the value of gamma and produces better performance when the gamma value is increased. They use Coiflet wavelets for the feature extraction.

3 Proposed Methodology

The Objective of this proposed methodology is to use CNN-SVM classifier for the classification of medical images into three classes: Brain, chest and Knee and further predict whether the classified image is suffering from brain tumor or not, COVID-19 or not and Knee OS or not respectively. A flow diagram of the proposed methodology is depicted in Fig. 1.

3.1 Dataset

The Three different datasets, which are available in Kaggle publicly were used in this study. The first dataset is called Brain Tumor Classification (MRI) to classify the MRI images into four classes, but in this study these four classes are converted into one class for the first classifier namely CNN and is labelled as Brain and into two classes for the 2nd classifier namely SVM labelled as Brain tumor or normal to serve the purpose of the proposed methodology. The total number of images in this dataset is 3264. The second dataset is called COVID-19 Chest X-ray Image Dataset to classify the images into normal and Covid-19 affected. For the 2nd classifier the classes remain the same and for the first the images are merged into one folder or class called the chest for CNN classifier to be trained and classify the image as chest. The total number of images in this dataset is less which is 94. Lastly, the third dataset is called Knee Osteoarthritis Dataset with KL Grading- 2018 which is to predict the severity grading of Knee OS. In this dataset there are total 5 classes having labels as 0, 1, 2, 3, 4 in which 0 represent the healthy knee and other classes represent the levels of the person affected by the knee OS. For the first classifier this dataset is merged into one and labelled as Knee and for the 2nd it is merged into two 0 as the normal representing person having healthy knee and 1, 2, 3 and 4 are merged into one and labelled as Knee OS showing being affected by the Knee OS. The total number of images in this dataset is 9786. Figure 2 shows the sample of the dataset.

Fig. 1.
figure 1

Flowchart of proposed CNN-SVM model

Full size image

3.2 Data Preprocessing

Getting Dataset

As discussed in above section that the dataset used is from Kaggle and for the purpose of processing the dataset they are arranged and kept in a folder. Firstly, this folder was downloaded into the google drive and with the help of function drive.mount the drive content is being loaded to our google collab environment, which platform was used to perform this study.

Importing Libraries

The programming language used to perform this study was Python and it is rich when comes to libraries. Hence all the necessary libraries were imported for preprocessing the data.

Fig. 2.
figure 2

Sample of medical images from dataset

Full size image

Importing Dataset

For importing the dataset os library was used. Some of the function used were os.listdir(), os.walk() and os.path.join().

Data Augmentation

Data Augmentation refers to making slight changes to the current data to expand its uniqueness without having to collect new information. It prevents a neural network from learning features that are not useful [16].

Normalization

Since the dataset was collected from three different dataset, so the format used by them were also different some files were of.png, some were.jpeg. To apply same algorithm to these datasets, dataset needed to be formatted in one format and this is known as normalization.

Image standardization

Since the images used have different heights and width, Image Standardization technique was used to make all the image of same height and width.

Splitting

The last step of data preprocessing includes splitting the data into training and testing (in this case validation) data. Split () function was used. Since the dataset was imbalanced so was the training and testing dataset after splitting.

3.3 Convolutional Neural Networks (CNN)

Fig. 3.
figure 3

The CNN model

Full size image

Among various deep learning models, the most commonly used model for image analysis is CNN model. The basic idea of what CNN model does is based on the meaning behind convolution, which is a type of linear operation in which two functions are multiplied to produce a third function that expresses how one function’s shape is changed by the other. Simply put, two matrices are multiplied to provide an output that is used to extract features from the image [17]. CNN model typically has two main parts: Feature extraction and Classification as shown in the Fig. 3. CNN model which is designed for image classification normally consist of four layers: Input layer, Convolution layer, Pooling layer, and Dense layer. Input layer, Convolution layer and Pooling layer are the part of feature extraction and Dense layer which is basically fully connected layers that receive inputs from convolution layer’s output and classify the image accordingly is the part of the classification.

In the proposed model, CNN classifier classifies the medical image among three classes. The classifier is sequential and consist of 1 input layer, 8 convolution layer, 4 max pooling, 2 fully connected layer, 1 dropout, 1 softmax and 1 classification layer. Since this classifier was built to classify given into three classes, the output layer has three neurons. The dropout layer in between the dense layers was added to reduce the problem of overfitting as it switches off some of the neurons that are focused on forcing the data finding the new paths. The activation function used was ReLU. From the last fully connected layer, which was a three-dimensional feature vector, as given as input to softmax classifier which is the most recommended when comes to the multiclassification, makes the prediction about organ type. The CNN classifier is then compiled with the Adam optimizer, categorical cross entropy loss function, and an accuracy metric. The model is then trained with the the dataset having 10 epochs and producing accuracy of 0.99 and loss of 0.012. The output of the Classifier is shown in Fig. 4.

The classifier was then saved using model. Save () function as “model_detection.h5” so that we can later use the model in the algorithm of the proposed model.

Fig. 4.
figure 4

Prediction result by CNN model

Full size image

3.4 Support Vector Machine (SVM)

SVM is an efficient learning method and a powerful supervised classifier. The core principle behind an SVM classifier is to select a distinguishing hyper-plane for the two classes that are separated by a threshold. It was developed for range of problems based on learning, prediction, and classification. It uses the concept of learning and kernel functions [18]. When it comes to pattern categorization, problems are rarely linearly separable therefore, nonlinear functions are used to translate input patterns to a higher dimensional space [19,20,21]. In the proposed work, we had made three SVM binary classifier for classifying whether the organ is affected with the respective disease. SVM model is trained using the images by converting the image into array form as shown the sample in Table 1 and then was being saved using pickle. Dump () function. The output is in the form of 0 and 1 showing 0 as not affected and 1 as being affected.

Table 1. Sample of array of different images
Full size table

4 Result

The proposed model successfully classifies and predict the medical image. For transparency, the output also shows the accuracy of the predicted image. The Output shows the image, name whether the image classifies as brain, chest or knee and then shows the accuracy of predicted image as being affected with the disease and the percentage it is predicted as normal and then at last gives the Output. Following figures shows the output of the proposed model. For the sample purpose 6 outputs are shown for 6 different possible inputs (Figs. 5, 6, 7, 8, 9, 10).

Fig. 5.
figure 5

Prediction result of covid medical imaging

Full size image
Fig. 6.
figure 6

Prediction result of chest normal medical imaging

Full size image
Fig. 7.
figure 7

Prediction result of knee OS medical imaging

Full size image
Fig. 8.
figure 8

Prediction result of knee normal medical imaging

Full size image
Fig. 9.
figure 9

Prediction result of brain tumor medical imaging

Full size image
Fig. 10.
figure 10

Prediction result of brain normal medical imaging

Full size image

5 Conclusion

This paper proposed a novel hybrid model CNN-SVM for the multiclass classification of three different disease brain tumor, Coronavirus and Knee OS using medical images. It breaks the problem into two parts, it first identifies whether the image is of which body part or organ and then predict if it has disease or not. Firstly, multiclass classification is done using CNN model and prediction is made whether the image is chest, brain or knee and then Binary classification is done using SVM classifier to predict whether it is affected with brain tumor, Coronavirus, Knee OS or normal (not affected with disease). The proposed model successfully able to classify and predict the images as shown in the Sect. 4. For the transparency purpose this model focuses on showing the accuracy percentage of the predicted image as this model is dealing with detection of disease where wrong prediction can have consequences. For the future work, this model can further be extended to classify the different types or severity grading of disease and not just classifying as organ being affected or not. More work can be done by improving the accuracy of each SVM model for even more better prediction.