Keywords

1 Introduction

Multiple sclerosis (MS) is a well-known autoimmune chronic disease which causes visuals, sensor and motor problems having as a direct consequence the deterioration of the functional status of the central nervous system (CNS). Besides, it is difficult enough to timely detect MS disease as there are no certain symptoms and physical findings that dictate its diagnosis. To this end, a multitude of medical approaches have been adopted in order to accurately assess and diagnose MS. To date, the most applicable medical treatment is magnetic resonance imaging (MRI) which is a non-invasive imaging technology that creates anatomical images.

1.1 Related Work

Most of the MRI research works for MS diagnosis are based on the implementation of machine learning (ML) or deep learning (DL) techniques in MRI scans in order to extract critical conclusions from brain images, [25, 26]. Although such type of processes is accurate and robust, it has turned out to be time-consuming and susceptible to manual errors, [21]. Instead, ML techniques have been rapidly evolved as the most promising player in the arena of MS decision support systems during the last decade. Such type of techniques does not require any prior knowledge or experience related to MS from clinicians facilitating the most accurate and objective diagnosis.

In particular, the most widely ML and DL-employed techniques have incorporated multiple data sources as input parameters such as clinical data, MRI scans, optical coherence tomography (OCT) data and motor evoked potential (MEP) measurements, [2, 18]. Some representative works will be presented in order to clarify the importance of ML models for MS decision support. To start with, [9]. In this paper the authors proposed a machine learning pipeline for clinical questionnaires analysis which aimed at detecting MS disease course. In particular, patient-reported outcomes (PRO) questionnaires were used in order to capture the self-perception of the MS disease. Besides, in a recent work [16], serum and CSF levels of forty-five cytokines were analyzed to identify MS diagnostic markers. Thus, cytokines were analyzed using multiplex immunoassay. Analysis of variance-based parameters and Pearson correlation coefficient scores were employed in order to utilize the appropriate input parameters for classification purposes. In the same context, [1], text mining methods were introduced in transcriptomic data analysis of multiple sclerosis disease for the first time. A complete predictive model was developed by taking into consideration consecutive transcriptomic data preprocessing procedures. Besides, the KmerFIDF method was utilized as a feature extraction method and linear discriminant analysis for dimensionality reduction. Additionally, in [17], a support vector machine (SVM) method with tenfold validation performed on specific properties of patients’ blood, such as zinc, adiponectin, total radical-trapping antioxidant parameter and, sulfhydryl, in order to predict MS with high sensitivity, specificity, and accuracy.

There are also published works which are not strongly dependent on medical data, but their analysis has been built from raw data such as gait disturbances [17] or exhaled breath analysis [8]. In both of the former works, four classification algorithms were employed in total: i) Logistic Regression (LR), ii) XGBoost (XGB) iii) SVM and iv) artificial neural network (ANN) model in order to analyze the imported raw data for MS prediction and classification purposes. Noteworthy, classifications and predictions have been enhanced by including the parameter of expanded disability status scale (EDSS), [14], which is a method of quantifying disability in multiple sclerosis and tacking down the evolution of the disability. Namely, it holds values from 0 (healthy person) to 10 (death). In this context, all of the following indicative published works, [12, 13, 27] have adopted the EDSS parameter as the target of their classification ML techniques. To this end, a multitude of ML techniques has been utilized, such as Bayesian, random decision trees as well as simple logistic-linear regression.

Apart from the EDSS parameter, there is a specific additional type of data that improved the accuracy and sensitivity of ML models. Such type of data is derived from MEP measurements, namely conducted measurements which quantify the conductivity of the CNS. In [5, 23], MEP measurements were carried out and were further analyzed by random forests and linear regression classifiers. To this end, the current study is based on MEP measurements and utilizes the EDSS as a target parameter for the employed ML algorithms. The required dataset regarding the MEP measurements has been derived from a recently published paper, [24].

It is worth noting that the evaluation of a patient’s disability is a multi-parameter medical process which is prone to EDSS miscalculation due to manual errors, and it is time-consuming as well. Thus, the estimation of EDSS through an automated analysis of a nervous system signal pulse, as suggested by the current work, could accelerate all the procedures in terms of MS prognosis and medical treatment.

The rest of the current research work is organized as follows: In Sect. 2, the structure of the used dataset and the metadata regarding patients is presented. The key features of the five employed machine learning models are described in the next Sect. 3 while the analysis of the derived results is developed in Sect. 4. Finally, the main conclusions and the proposal for future work are given in the last section.

2 Description of Dataset

The dataset derived from the work of Yperman et al., [24], contains data regarding electrical signals which have propagated through the NS and detected from either hand or foot. In particular, the brain of each patient has been stimulated through a magnetic stimulator and an external trigger system leading to the creation of a signal pulse which propagates along the nervous system. Samples of the resulting signal are detected, are stored and exported into a file. To be more specific, 2000 time points in a time window of 100 ms determine the signal shape. In total, the dataset includes information about:

  • metadata of patients (963 records) such as age, gender, time of hospital visit, type of machine that conducted the measurements, teams that carried out the MEP measurements, etc. For more information, see [24].

  • MEP measurements (96290 records).

  • EDSS values (7414 records) for specific patients.

As it has already been mentioned, the target of the ML techniques is the EDSS value for each patient at the specific time of MEP measurement. To achieve this, critical properties of the resulting signals should be taken into consideration in order to be used as input parameters to the machine learning models. The following table describes all the input parameters used for the development and analysis of the suggested prediction model.

Table 1. Description of the utilized input parameters.
Full size table

3 Machine Learning Models

The following models for approximating Features-Target relationship have been employed.

  1. 1.

    Linear Regression as a baseline model for the next ones.

  2. 2.

    Polynomial Regression. As it is necessary to select from a vast pool of potential nonlinear features along with their number, the ITSO [4] as well as PROS [19] Optimization Algorithms for Feature Selection have been adopted, which have also been found experimentally vastly efficient.

  3. 3.

    Gradient Boosting [7, 10, 22] with hyperparameter tuning. Particularly, the grid search method with cross-validation has been utilized.

  4. 4.

    Random Forests [6], as implemented in [20]

  5. 5.

    Artificial Neural Networks [3].

For each model, the following computations are carried out:

  • The accuracy among the Prediction and Target-variable for the Train and Test Sets.

  • Error analysis: Residual Errors vs Target diagrams, Probability Density Functions, Cumulative Density Functions, for the Train and Test Sets as well.

The former approach is beneficial for detecting specific patterns occurring in the prediction and, hence, enhancing the generalization capability and reliability of the model.

4 Results

The application of the aforementioned ML techniques produced a vast amount of constructive results which shed light on the MS disease. It should be highlighted, that input parameter with high discrepancies or abrupt deviations of the signal voltage magnitude in the time domain has been removed from the calculations. In the same, context, all the spikes occurred at the beginning of the detected pulse have been deleted. Figure 1 shows the pairwise correlations among the input parameters. Particularly, we reform the correlation matrix, such that the non-zero values are concentrated around the diagonal of the matrix. This is performed with the CuthillMcKee Method [11, 15]. This way, the clusters of associated features are computationally identified.

Fig. 1.
figure 1

CuthillMcKee representation of the input parameters, see Table 1. Note that the x-axis label is exactly the same as that of y-axis. For the sake of readability, the x-axis labels are described with numbers which correspond to specific input parameters, as shown on the left side of the graph.

Full size image
Fig. 2.
figure 2

Target vs predicted EDSS derived from linear model.

Full size image
Fig. 3.
figure 3

Target vs predicted EDSS derived from XGBoost.

Full size image
Fig. 4.
figure 4

Error metrics for 5 machine learning methods implemented in the current analysis.

Full size image
Fig. 5.
figure 5

Pearson correlation along the number of samples for the XGBoost model.

Full size image

4.1 Performance of ML Models

After an exhaustive search for the hyperparameters of the studied ML models, we identified that RF and XGB exhibited the best performance (Figs. 2, 3). The error metrics are depicted in Fig. 4. Although the final models have adequate accuracy, some errors occur, especially for lower values of EDSS. Hence we run a data adequacy check (Fig. 5), where it is demonstrated that the accuracy is increased with the number of training samples.

4.2 Sensitivity Analysis

Finally, we run a sensitivity analysis of the input parameters with respect to the target, by keeping all features constant to a particular value (median, 25% and 75% quantiles), and change the studied feature within its given values in the initial dataset. Accordingly, we predict using each one of the models. Figures 6 and 7 illustrate that the Anatomy AH, when it is equal to the unit, the EDSS results in lower values. Furthermore, Figs. 8 and 10 depict an increasing pattern of EDSS with respect to age.

Fig. 6.
figure 6

Sensitivity analysis for AnatomyAH=1 based on the RandomForests model.

Full size image
Fig. 7.
figure 7

Sensitivity analysis for AnatomyAH=1 based on the XGBoost model.

Full size image
Fig. 8.
figure 8

Presentation of increasing pattern between the input parameter of age and EDSS emerged from the RandomForests model.

Full size image
Fig. 9.
figure 9

Features importance derived by XGBoost model, which was proved to provide the most accurate prediction results.

Full size image
Fig. 10.
figure 10

Resentation of increasing pattern between the input parameter of age and EDSS emerged from the XGBoost model.

Full size image

5 Conclusions

The present research analysis in concert with former descriptive Figs. 1-3 manifest that the highest Pearson correlation was achieved through the implementation of XGBoost, reaching approximately 96% accuracy. On the contrary, the lowest accuracy of 58% was obtained by the artificial neural network. Thus, the most appropriate ML model to predict the value of EDSS from a propagating signal on the nervous system is XGBoost.

One of the most critical conclusions drawn in the current MS analysis is depicted in Fig. 9 wherein the most effective input parameters of the prediction model are presented. In particular, the age, gender, anatomyAH and maximum peak are the prominent ones. Among the former significant input parameters are also the energy, time difference of local peaks as well as the time points at either local maximums or minimums occur.

Besides, the Pearson correlation follows an increasing linear trend along with the increase of input training data, Fig. 5. Hence, the suggested prediction model can be clarified as efficient and robust.

The early results of the current research work dictate its extension to the prediction of the forthcoming evolution of EDSS for a certain MS patient. This is of great importance as it will determine the most appropriate long-term medical treatment to enhance the quality of patient’s life.