Introduction

Osteoarthritis is a usual erythrogenic disease that aims at the joints of the body, especially the hips, knees, hands, and fingers. The knee also called as hinge joint comprises of bones, cartilage, ligaments, tendons, and tissues. Cartilage is a smooth, strong, and elastic tissue that secures the ends of the bone thus gliding the bones while walking. In osteoarthritis, the cartilage constantly wears away and becomes ragged. The protective space between the bones decreases resulting in bone kneading the other and producing painful bone spurs, with which one may feel the loss of body control and walking uncomfortable. The major forewarnings of Knee Osteoarthritis (KOA) may be pain, tender, and puffy joints, stiffness after extended rest, obstructed joint functioning, etc., these evidence along with clinical examination and radiographic images are used by the medical practitioners to examine the progress in knee OA [17]. The element findings of OA in X-rays are bone eburnation, decomposition of joint cartilage in terms of joint space, bone spurs, genesis of subchondral sclerosis and cysts. Figure 1 shows the normal knee and osteoarthritic knee.

Fig. 1
figure 1

Normal knee and osteoarthritic knee [31]

Full size image

Knee OA is classified by Kellgren–Lawrence (KL) system into five grades [13, 14] as shown in Table 1. Each grade exhibits different symptoms and are treated separately.

Table 1 Five KL grades that classify OA
Full size table

World Health Organization (WHO) reports that the prevalence of OA increases on consequences of population aging, obesity, joint hypermobility, valgus/varus posture, fracture along the articular surface, congenital defects, immobilization, family history, and metabolic causes [32]. It is valuated that 10–15% of the world's population suffers from some degree of OA. Roughly 13% of women and 10% of men, 60 years and older have symptomatic knee osteoarthritis. Patients with knee OA have low apprehensions of their quality of life, functional limitations, and pain, thus leading a sedentary lifestyle and decreasing life span by 4–10 years. As there is no proven remedy for this circumstance, it is very needful to track the degree of the disease and set up more considerable methods to assess knee OA progression. Among the many diagnostic methods, radiographs seem to be easily reachable and affordable. So in this study, we have worked on radiograph images.

Related Work

The present-day artificial intelligence systems are being extensively used in medical fields, be it in diagnosing the patients or designing the right treatment plan by the medical practitioners based on the outcome predictions. Convolutional Neural Networks consist of convolutional layers and pooling layers, which are excellent at learning the most important features of images and classifying them into multiple classes more precisely and quickly. A combination of techniques like digital image processing, pattern recognition, and machine learning as well, can help in diagnosis [7, 10, 21, 24].

To enhance medical visualizations many authors have contributed, focusing more on X-ray images [3, 17] amongst many of the other modalities like MRI [25], and CT images [22, 23]. Clinically collected raw data or data from the standard dataset [6] are considered as input. First, the data are pre-processed, i.e. denoising using suitable filters [15, 18] like circular Fourier Filter [4], edge detection techniques [2] like Sobel, Prewitt, Robert, Zero Cross, and Canny Edge detection algorithms [16, 21, 24], contrast enhancement, histogram equalization, thresholding, and canny edge detection [4]. Using these images some authors have built a fully automated system [1, 4, 19] and some semi-automated systems. While developing such systems many techniques like machine learning algorithms [4] with a classification rate of 82.98%, and feature extraction [20] with an accuracy of 66.6% are used. Some of the other techniques are feature extraction techniques like Haralick [29], Fisher score [26] with accuracy of 91.5%, wavelets [9], different image segmentation [16, 19] like Watershed, Otsu's segmentation [28], ASM [15], pixel segmentation [30], thresholding, region-based, clustering, Markov random field, artificial neural network, deformable and atlas guided method [25]. Some used cartilage maps for segmentation [27], morphological techniques [1], DenseNet [5, 18]. Some used classification techniques like Naive Bayes, random forest classifiers [4], feature-based classification [12], ANN [8, 11, 19], SVM [29], KNN [28] with cartilage classification rate of 78.2% and tibial cartilage classification rate to be 82.6%.

From this literature, it is clear that many authors have carried out their research for varied imaging techniques to detect and assess KOA. Some papers give precise outcomes for grades 2 and 3 but imperfect outcomes for grade 1, others focused on different techniques used and gave a comparison [25]. But it is also noticeable that advancements in machine learning can still be investigated to meet up the challenges. Thus in this paper, our motive is to use the latest development in Neural networks, i.e., DenseNet that will extract required features from the knee radiographs, automatically diagnose the severity of KOA and give close and unambiguous results.

Architecture of DenseNet

DenseNet-201 is one of the convolutional neural networks with 201 layers deep which gives improved accuracy and is better and more efficient in training. Network with pretrained version uses millions of images and more from the ImageNet database for training which will classify images into 1000 object classes. This helps the network learn greater feature representations for an extensive range of images. The network accepts an image input size of 224-by-224. A pretrained DenseNet-201 CNN architecture returns a DAGNetwork object, while an untrained DenseNet-201 CNN architecture, returns a LayerGraph object. Figure 2 shows the architecture of DenseNet with three Dense Blocks.

Fig. 2
figure 2

DenseNet architecture with three dense blocks

Full size image

The traditional convolutional network having N layers connects every layer with its subsequent layer using N connections. Dense Net creates N(N + 2)/2 direct connections. DenseNet architecture concatenates the features from a series of previous layers in a feedforward manner and also takes care that each convolutional function generates the same number of features. DenseNet consists of multiple Dense Blocks and each Block consists of a Convolutional Layer and Pooling Layer [18]. The convolutional layer extracts features by sliding the kernel over the input image. The pooling layer helps in the dimensionality reduction of feature maps. These dense blocks are blended to learn from previous layer features, keeping the count of learning parameters to be low to avoid overfitting the model. DenseNet has mastery in building feature propagation, helps feature reuse, reduced the number of parameters, better gradient/information flow in the network, and is easy to train.

Design of OA Detection Using Simple Radiographs

Figure 3 gives the flow diagram of the proposed method to detect OA severity using simple radiographic images.

Fig. 3
figure 3

Flow diagram of the proposed method

Full size image

Input Image

Dataset used: Input images are used from the Osteoarthritis Initiative (OAI) dataset. The OAI [6] is a multi-center, continuing, proposed, 10-year experimental study of radiographs of both genders, sponsored by the NIH with a goal to investigate natural history and provide better awareness of the prevention and treatment of KOA. It is a public archive of data, joint images, and biological samples gathered over time from two different populations of subjects, one with Knee OA at the baseline and the other at risk of developing knee OA. It contains 9786 knee images from 4796 participants ranging in age from 45 to 79 and is classified as per KL grades. Here, AP view images in PNG format are used as input. This dataset stores the radiographic knee images in files according to the KL grades, with a varied number of images for each grade. Raw clinical images can be used. This process called acquisition is a procedure to get the input image, which when processed gives finely detailed physical attributes in terms of image characteristics that calibrate image processing to get adaptable interpretations of the final display.

Split

 In this work, as reported in the OAI dataset, a total of 5778 out of 9786 images are used for training which is categorized into five grades, viz. Grade 0, 1, 2, 3, and Grade 4. Table 2 shows the number of images available in the standard dataset for training.

Table 2 Images in the standard dataset
Full size table

For every experimentation, to consider the equal distribution of images, 170 images are randomly chosen from each of the Grades. Of these 170 images, 80% were used for training and the rest for testing.

Pre-Processing

The first element is the image input layer, which accepts input images of size 224-by-224-by-3 and 3 being the number of color channels. Thus, pre-processing step converts the input image to grayscale, crops and resizes the image to 224 × 224 pixels; however, any size can be considered.

Feature Extraction Using DenseNet 201

DenseNet contains dense blocks and each dense block comprises of convolutional layer, ReLU, and batch normalization. Two such dense blocks are connected with convolutional and max-pooling layers. The final dense block is connected to a global average pooling and softmax classifier. Once the data is loaded the dense convolutional layers extract image features, then replace the final layers and freeze the initial layers, with which the network is thus trained. This forms a fully connected layer as the last layer with learnable weights and replaces the number of outputs with several classes specified by the classification layer. The input features are stored in overall spatial locations with the help of a global pooling layer. Thus, the layers and connections are extracted from the layer graph and layers to freeze are selected. Later these are reconnected in original order with earlier layers set to zero. Training options are now specified like a learning rate of 3\(e^{ - 4}\) for 40 epochs on NVIDIA GEFORCE X GPU with compute capability of 3.14.0.139. This implementation requires any i5 processor or above, a minimum of 4.00 GB installed RAM, and is coded in MATLAB R2020b. The validation images are passed through the fine-tuned network and classification accuracy is calculated.

Image Classification

The algorithm now predicts the class labels for the input knee X-ray in the datastore into five KL grades using the trained network.

Here classification layer is fully connected to the network. Dimensions of feature maps are reduced using global max pooling. Later probabilities of classes are computed by using a softmax activation function on the feature maps. The highest probability class gives the output value class. The softmax function is specified as in Eq. (1).

$$P\left( {x_{i} } \right) = \frac{{e^{{x_{i} }} }}{{\mathop \sum \nolimits_{i\varepsilon K} e^{{x_{i} }} }},$$
(1)

where denotes the probability score of and is the vector input value of K real numbers. After calculation sixteen sample validation images along with predicted labels and their probabilities are displayed.

Results and Discussion

We aim to provide a non-invasive, automatic method to detect knee OA severity using radiographic images. Considering the clinical pieces of evidence from knee X-ray images, the DenseNet model classifies the knee into five KL Grades, namely Grade 0, Grade 1, Grade 2, Grade 3, and Grade 4.

From the OAI standard dataset, 170 images each were randomly selected from every grade of knee osteoarthritis and saved, respectively, as Grade 0 through Grade 4. The 170 images are randomly divided into 80/20 split, ensuring 136 training and 34 testing images. For testing, any random image is chosen from the dataset and is tested. The model is trained with 10 to be the mini-batch size, 40 epochs, with 85 iterations each epoch, all giving us 3400 maximum iterations. After completion of the training process, the model is tested, which exhibits the accuracy for each grade as shown in Table 3. This model is validated by medical experts and is found to be accurate.

Table 3 Accuracy calculation in each KL Grade
Full size table

Figures 4 and 5 show that the trained model has correctly classified the test radiographic images as per KL Grades, with appropriate class names, and also calculates the accuracies for each of the grades.

Fig. 4
figure 4

Confusion matrix for classification into KL grades

Full size image
Fig. 5
figure 5

Image classification with accuracy

Full size image

Norman et al. [18] have worked on 4905 knee radiographs and have got the accuracy to be 91.52%. We have used 850 knee radiographs randomly selected and have got an accuracy of 91.75%, so with more radiographs, we can attain better results.

The radiographic images reveal evidence of knee osteoarthritis, but often with low resolution. This is a challenging issue of noise removal and image enhancement that may improve performance. From the above table, it is evident that the system can classify Grade 0, 3, and 4 with better results as compared to Grade 1 and 2. As pointed out by medical practitioners, this is because Grade 1 and 2 have similar clinical findings with a slight increase in severity of Grade 2. Hence, future research may investigate different parameters noticeable from knee X-ray images that can help model training with optimal learning parameters.

Conclusion

This study reviews the varied transformations in the field of machine learning like CNN uses deep learning to examine radiographic images, as manual assessment is difficult to identify insignificant progression in early detection of knee osteoarthritis. Radiographic Images are easily available and the first step prescribed by the medical practitioner to detect KOA. The present work uses DenseNet neural network and knee X-rays to develop an automatic knee OA classifier. This model has shown better results in classifying the input knee X-ray images into five KL grades of severity. The overall accuracy of the work was found to be 91.75%, which is comparable with existing methods. Thus, with the increased number of X-ray images, this work helps to assist medical practitioners in diagnosing precisely and error-free detecting the severity of the disease, which aids in quick treatment to keep the patients to be in comfort.