Background

Ischemic stroke ranks as the second most prevalent cause of disability and mortality on a global scale, leading to significant financial burdens in healthcare expenditures, as well as productivity and economic setbacks [1, 2]. Approximately 17.8 million adults in China are estimated to have suffered a stroke in 2020 [3]. The utilization of recombinant tissue plasminogen activator (rt-PA) has substantially enhanced the effectiveness of treating acute ischemic stroke (AIS) [4]. However, a considerable portion of patients continue to experience unfavorable functional outcomes or mortality [5], commonly referred to as “thrombolysis resistance (TR)”, also known as “non-response to thrombolysis”, or “ineffective thrombolysis” or “poor response to thrombolytic therapy”. TR is a complex clinical phenomenon characterized by the persistence of thrombus despite standard thrombolytic therapy. It has been shown that platelet-rich thrombi had TR [6]. Extracellular traps of neutrophils are expected to be a new targeting measure for TR [7]. Addressing the TR necessitates a thorough examination of multiple factors and the implementation of personalized treatment strategies. However, TR is one of the factors that affect the prognosis of AIS, with other factors including blood pressure and subsequent severe brain damage, etc. [8, 9].Therefore, the development of clinical prediction models is of paramount importance in optimizing patient selection and outcomes in intravenous thrombolysis (IVT) treatment. Currently, there is no uniformity in judging the effectiveness of thrombolysis, and previous studies have focused on the use of multimodal imaging techniques [10], However, due to the strict treatment time window and relatively complex equipment requirements, it is difficult to promote and popularise it in primary hospitals. It is particularly important to find a scientific and convenient method to predict the effect of thrombolysis and prepare the bridging treatment after thrombolysis failure in advance.

Machine learning (ML) algorithms, known for their ability to optimize information for enhanced prediction accuracy, are becoming more prevalent in the field of medicine for quantifying risk, identifying predictors, and creating precise predictive models for diagnosis and prognosis [11, 12]. In this research, a predictive model for the risk of thrombosis was developed using machine learning techniques, aiding clinicians in timely preparation of high-risk patients for bridging endovascular treatment (EVT) and ultimately improving patient outcomes.

Methods

Patient cohorts

Patients diagnosed with AIS at our institution between January 2021 and December 2022 were selected for this study. All patients received rt-PA intravenous thrombolysis within 4.5 h of the onset of symptoms, and were not bridged for endovascular treatment. The inclusion criteria were as follows:1. The clinical manifestations and imaging data of the patient were consistent with acute ischemic stroke [13]; 2.Age ≥ 18 years; 3.Complete clinical data and signed the informed consent for intravenous thrombolytic therapy and enrollment;4. The Ethics Committee of our hospital has approved this study, and followed the Declaration of Helsinki. The exclusion criteria were as follows: 1.Haemorrhagic cerebrovascular disease, mixed cerebral infarction; 2.Other organic diseases of the nervous system, such as encephalitis, tumour, traumatic brain injury, etc.; 3.Accompanied by other serious systemic diseases, such as recent serious infections, previous diseases of the haematological system, etc.; 4.Mental disorders or consciousness disorder after the onset of the disease and unable to cooperate with the examination (Fig. 1).

Data preprocessing

The variables identified by the box plot that contain outliers are excluded.In addition, for variables with a missing rate of less than 5%, we will impute the missing values using the mode for the median for continuous variables. For the sensitivity analyses, we pursued multiple imputation using the multivariate imputation by chained equations method.

Data collection and variable selection

Given the limitations of the sample size, univariate logistic regression was initially performed on the collected variables. Any variables exhibiting non-statistically significant differences were subsequently excluded.

Clinical data: gender, age, hypertension (receiving medications for hypertension or blood pressure ≥ 140/90 mmHg on repeated measurements) [14]; diabetes mellitus (receiving medications for diabetes mellitus, fasting blood sugar ≥ 126 mg/dL or HbA1c ≥ 6.5%, or a casual plasma glucose > 200 mg/dL) [14]; atrial fibrillation (AF ): receiving medication for the treatment of AF or sustained episodes of AF heart rate > 30 s recorded by 12-lead electrocardiogram (ECG), single-lead ECG, or Holter monitor [15]; Systolic blood pressure (SBP), diastolic blood pressure (DBP), and door-needle time (DNT) were collected at admission. National Institutes of Health Stroke Scale (NIHSS) [16] at admission and 24-hours NIHSS after thrombolysis. Note: NIHSS at admission and 24-hour NIHSS were assessed by the same physician.

Laboratory data were pre-thrombolytic indicators, including blood glucose (Glu), white blood cell count (WBC), neutrophil count (Neu), lymphocyte count (Lym), monocyte count (Mon), platelet count (PLT), D-dimer, fibrinogen (FIB), creatinine (Cr), blood urea nitrogen (BUN), total cholesterol (TC).

Definition of functional outcome.

Thrombolysis resistance was defined as ([NIHSS at admission − 24-hour NIHSS] × 100％/ NIHSS at admission) ≤ 30% [17].

Model development and evaluation

The data were randomly divided into training and testing groups of 70% and 30%. We utilize the training group to ascertain the suitable parameters of the model and assess their efficacy by employing the testing group. There exist numerous approaches to machine learning, yet a definitive conclusion regarding the most effective one remains elusive. Five machine learning algorithms were used in this study: logistic regression (LR), the least absolute shrinkage and selection operator (LASSO), extreme gradient boosting (XGBoost), support vector machine (SVM), and random forest (RF), similar with previous study [18, 19]. This study employs a 10-fold cross-validation approach for model derivation, and utilizes a combination of the grid search algorithm to optimize the model hyperparameters. To evaluate the performance of each model, the features derived from the training group are applied to the testing group. In this study, the receiver operating characteristic (ROC) curves of the five models were plotted and the performance of the models was evaluated based on the area under the curve (AUC), specificity, sensitivity, accuracy, positive predictive value (PPV), negative predictive value (NPV), precision, recall, and F1 value. Constructing a nomogram based on the output of the most influential variables from the best model. Finally, the accuracy, calibration, and clinical applicability of the model were evaluated in the testing group using ROC curves, calibration curves, and decision curve analysis (DCA) (Fig. 1).

Statistical analysis

Values are presented as the mean ± standard deviation, median(interquartile range) for continuous variables, or as the number (%) of subjects for categorical variables, as appropriate. Comparisonsof the characteristics between the two groups were performed by the t-test, chi-square test, Mann-Whitney U parametric test according to the type of the variable. The statistical significance was defined as a two-tailed P-value of < 0.05. Analyses were performed in SPSS (version 26, IBM, New York, NY, USA) and R (version 3.3.2; R Project for Statistical Computing) using the Caret ML library.

Results

Characteristics of study population

According to the inclusion criteria 232 patients with AIS treated with intravenous thrombolysis were selected and finally 218 patients were included after excluding 14 patients according to the exclusion criteria (Fig. 1). The median age was 68 years, 149 (68.3%) were male, 135 (61.9%) had hypertension, 46 (21.1%) had DM, 19 (8.7%) had AF, and 88 (40.4%) developed thrombolysis resistance. The study population was randomly divided into training (n = 152) and testing (n = 66) groups on 7:3 basis, and there was no statistical difference in the data characteristics between the two groups (P > 0.05) (Table 1).

Table 1 Clinical characteristics of the training and testing groups
Full size table

Prediction model performance

In this study, five models, including LR, LASSO, SVM, XGBoost and RF, were established, and the ROC curves of the five models were constructed (Fig. 2). In the training group, it can be seen that the LASSO model has the largest AUC (0.789), and its specificity of 0.777, PPV of 0.672, accuracy of 0.763, precision of 0.672, and F1 value of 0.705 are better than the other models; In the testing group, the LASSO model exhibited superior performance compared to other models in terms of AUC (0.765), specificity of 0.694, PPV of 0.676, accuracy of 0.727, and precision of 0.676. Consequently, LASSO is considered to have good overall performance and selected as the best model (Fig. 2 and Table 2).

Fig. 1
figure 1

Flowchart of the study population and process. LR, logistic regression; LASSO, the least absolute shrinkage and selection operator; SVM, support vector machine; XGBoost, extreme gradient boosting; RF, random forest; ML, machine learning

Full size image
Fig. 2
figure 2

Comparison of the receiver operating characteristic (ROC) curves for machine learning models in which thrombolysis resistance occurs. (A) Training group. (B) Testing group.LR, logistic regression; LASSO, the least absolute shrinkage and selection operator; XGBoost, extreme gradient boosting; SVM, support vector machine; RF, random forest; AUC, area under the curve

Full size image
Table 2 Statistical comparison of machine learning models for the occurrence of thrombolysis resistance
Full size table

Figure 3A shows a plot of the coefficients of the LASSO variables. Each curve in the graph represents the trajectory of the coefficient of an independent variable. The vertical coordinate is the value of the coefficient. The lower horizontal coordinate λ is the parameter controlling the severity of the penalty. The upper horizontal coordinate is the number of non-zero coefficients in the model under the penalty parameter. Figure 3B shows the tuning parameters in the LASSO model, filtered by 10-fold cross-validation. Vertical dashed lines are plotted at 1 standard error(SE) of the minimum and maximum criteria (1-SE criterion) to select the optimal λ value for the lASSO model. The 20 features were reduced to 5 potential predictors of the occurrence of thrombolysis resistance according to the LASSO method.

Fig. 3
figure 3

Feature selection based on LASSO model. (A) Identification of the optimal penalization coefficient lambda (λ) in the LASSO model. (B) Cross-validation for tuning parameter selection in the LASSO model. LASSO, the least absolute shrinkage and selection operator

Full size image

Nomogram

A nomogram of thrombolysis resistance was constructed by screening the five optimal predictor variables, including NIHSS at admission, GLu, WBC, Neu, and BUN (Fig. 4). Each predictor variable was represented by a vertical line on the value axis, with the intersection of the line and scoring axis indicating the variable’s score. The predicted probability of thrombolysis resistance was determined by summing the scores of each risk factor and projecting the total on the predictive probability axis. As the values of the 5 variables increase, the total score will become higher, the 5 variables are positively associated with a high risk of thrombolysis resistance.

Fig. 4
figure 4

Proposed nomogram for thrombolysis resistance. NIHSS, national institutes of health stroke scale; Glu, glucose; WBC, white blood cell count; Neu, neutrophil count; BUN, blood urea nitrogen

Full size image

Internal validation of the predictive model

The AUC values for the training and testing groups were 0.789 and 0.765, respectively. This suggests that the model has a reasonable ability to discriminate between subjects with thrombolysis resistance and those without it (Fig. 5A). DeLong’s test was used to statistically compare the AUCs of the training and testing groups. The P-value of 0.725 is greater than the significance level of 0.05, indicating that the difference in AUCs between the two groups is not statistically significant. This suggests that the model’s predictive accuracy is consistent across different datasets. The performance of the Calibration curve was internally validated by Bootstrap resampling (1000 times) analysis, The Hosmer-Lemeshow goodness-of-fit test was used to assess how well the model fits the observed data. P-values of 0.1225 for the training group and 0.4124 for the testing group, both greater than 0.05, indicate that the model fits the data well in both groups, which showed that the nomogram predicted the probability of thrombolysis resistance in good agreement with the actual probability. (Figure 5B and C). DCA were plotted to evaluate the clinical applicability of the model by comparing the net benefit of the model’s predictions against a range of threshold probabilities. The positive net benefit in both the training and testing groups (Fig. 5D) indicates that the model’s predictions provide a clinical advantage over a strategy of not using the model, suggesting that the model has good clinical utility.

Fig. 5
figure 5

Internal validation diagram for the model. (A) ROC curves for the training and testing groups; (B) Calibration curve for the training group, P = 0.1225; (C) Calibration curve for the testing group, P = 0.4124; (D) DCA for the training and testing groups

Full size image

Important variables of the machine learning models

After evaluating the significance of each feature, the 5 machine learning algorithms have ranked the top 5 variables based on their discriminatory performance. LR, LASSO, RF,SVM and XGBoost can estimate the contribution of each feature to the model by calculating the absolute value of the standardized regression coefficient, Gini coefficient, weight coefficient, and Gain value ( Table 3). NIHSS at admission and Glu were among the top 5 variables in all models simultaneously, with Neu appearing in 4 models. The results of LASSO and RF were comparable, but only 1 variable differed, and WBC was identified as an important variable in both. Blood pressure-related indicators were found to be significant characteristics in LR and SVM. FIB was identified in XGBoost, RF, while BUN only appeared in LASSO.

Table 3 Top 5 important features in each models
Full size table

Discussion

Currently, numerous studies have referenced about thrombolytic effect, but there is no precise definition or specific machine prediction model research for TR. In this study, the term “TR” was proposed based on the early thrombolysis effect in AIS patients, and presents a risk prediction model for identifying patients at risk for thrombolysis resistance by integrating easily accessible clinical variables with machine learning algorithms.

In clinical studies, there is a lack of standardization in the criteria used to evaluate the efficacy of early thrombolysis in patients with AIS. The modified Rankin Scale (mRS) scores and NIHSS score are two commonly used stroke assessment tools. The mRS score commonly used as the primary efficacy endpoint in clinical trials for its simplicity and ease of interpretation [20]. The mRS scores are typically assessed 3–6 months after thrombolysis and are susceptible to a number of factors, and the changes in 24-hour NIHSS scores were chosen in this study as a potentially more effective way to assess the early effects of thrombolysis. Some studies have used a decrease of ≥ 4 points or an increase of ≥ 4 points in the NIHSS from baseline to 24 hours after intravenous thrombolytic therapy to assess the early therapeutic effect of thrombolysis in patients [21, 22]. This study concluded that a 4-point decrease or increase in admission NIHSS for patients with different admission NIHSS does not represent the same degree of improvement or deterioration, and that the use of percentage change in NIHSS better explains the wide variation in patients’ admission scores and more accurately determines the effectiveness of thrombolysis [23]. A study on mechanical thrombolysis found that > 30% improvement in NIHSS score at the end of thrombolysis was a reliable prognostic predictor [24]. Meanwhile, data from a multicentre, prospective trial of intravenous alteplase for the treatment of AIS showed that there was no difference in the change in baseline versus 24-hour NIHSS difference between patients in the alteplase-treated group compared with the placebo group, whereas the change in the 24-hour NIHSS percentage was more pronounced, and 30% was the optimal cut-off value [17]. Consequently, TR was considered to be defined in this study as an improvement in NIHSS score of ≤ 30% from the admission NIHSS 24 hours after thrombolysis.

A total of 218 patients were included in this study, and 88 (40.4%) patients developed thrombolysis resistance. We conducted TR prediction models using ML algorithms, including LR, LASSO, XGBoost, SVM, and RF. Among the five models, LASSO was rated as the best model with superior AUC, specificity, PPV, accuracy, and precision. LASSO is based on a regression analysis method that does not rely on statistical significance for regularisation, but rather narrows down the coefficients of complex experimental variables and excludes relatively unimportant variables [25]. A Chinese study based on the GEO database to screen for differential genes developed a LASSO regression model to better identify the prognosis of AIS, and the AUC of its LASSO regression model was 0.969 [26]. A retrospective case-control study identifying AIS based on LASSO regression indicated that the model had a better identification ability (AUC = 0.916) [25]. The LASSO regression model in this study had an AUC value of 0.765, which was a good predictor of whether thrombolysis resistance occurred in patients with AIS, suggesting the potential utility of LASSO regression in predicting disease prognosis or diagnosis. We found that the sensitivity value of 0.767 in the LASSO model is slightly lower than that of the RF model (0.800) (Table 2). RF model is typically capable of handling more complex data patterns and possess a certain degree of resistance ato overfitting. Combining the AUC results, we thought the LASSO model is better than RF model. However, it is necessary to investigate this results with a large amount of data in the future study.

The nomogram (Fig. 4) made by the best model in this study showed that NIHSS at admission, Glu, WBC, Neu and BUN were significant predictors of the occurrence of TR. Several studies have confirmed that these five metrics are associated with poor prognosis in AIS patients [27,28,29,30,31], which is consistent with our findings, indicating that our model has scientific validity and credibility in real-world practice. A machine-learning based multicentre retrospective study concluded that the key features predicting functional outcome in AIS contain NIHSS at admission, and white blood cell count [32]. Meanwhile, a predictive model examining ineffective recanalisation in AIS considered NIHSS score, blood glucose, c-reactive protein, and creatinine as important predictors [33]. It suggests that inflammatory response may be a cruvial factor in the development of AIS [34]. Previous studies on Cr in the model and on BUN in the present study has primarily focused on renal function, with findings indicating that elevated BUN levels upon admission may indicated haemodynamic deterioration [35], a known predictor of poor prognosis and death in AIS [36]. In this study, it was hypothesised that it might be related to the shift of blood to vital organs such as heart and brain during the onset of AIS, resulting in a decrease in blood flow to the kidneys. In contrast, the role of blood glucose as a predictor of thrombolytic prognosis may be related to the fact that hyperglycaemia increases platelet mitochondrial reactive oxygen species production, which enhances procoagulant platelet formation and significantly increases infarct size [37]. Moreover, It is important to consider the influence of variables such as FIB, blood pressure-related markers, and TC, which have been included in other models, on the phenomenon of TR in this study. A predictive model on the prognosis of intravenous thrombolysis in patients with mild AIS identified higher NIHSS scores, DM, and deep infarcts in the middle cerebral artery (MCA) as potential predictors of a poor prognosis [38]. These findings align with those of the present model, which incorporates imaging data. For healthcare institutions lacking magnetic resonance imaging, the variables included in the LASSO model are more accessible, and patients at high risk for thrombolysis resistance are promptly transferred to hospitals equipped for endovascular therapy for further treatment. Yang H et al. [39]. used LASSO regression to construct the prognosis of patients with AIS found that NIHSS, SBP, lymphocyte percentage (LYM%), neutrophil-to-lymphocyte ratio (NLR), platelet-to-neutrophil ratio (PNR), and platelet-to-lymphocyte ratio (PLR) independently predicted early neurologic deterioration (END), which, after the incorporating the composite inflammatory index improved the AUC of the test set from 0.635 to 0.816. This study did not include the composite index for modeling for the time being, which is a good direction for research.

All the variables considered in this model are readily available at the time of admission, and when the values of the above five variables are entered into this model, clinicians will be able to quickly assess the effectiveness of thrombolysis in patients with AIS in advance, and decide whether to perform IVT/EVT or not. Limitations: There are numerous machine learning methods available, this study only focuses on 5 specific algorithms. To increase the stability of the model, additional machine learning algorithms could be incorporated. Additionally, The causes of poor functional prognosis or death after thrombolysis cannot be explained by a single theory, rather, they may be the result of a combination of factors. The study population could be expanded, more variables could be selected to build the model, and the model could be prospectively validated in an independent cohort. Furthermore, the definition of “thrombolysis resistance” proposed in this study has yet to be generalized to other studies. Additional research is necessary to verify the scientific validity and effectiveness of this definition.

Conclusion

In this study, we used a machine learning approach to screen five optimal predictors of thrombolysis resistance and constructed a nomogram to predict the risk of thrombolysis resistance.Among them, higher NIHSS at admission, WBC, Glu, Neu, and BUN were the risk factors for the occurrence of thrombolysis resistance. This model has the potential to assist clinicians in the formulation of personalized treatment plans. Conducting external validation to evaluate the broader applicability and to detect any possible biases is a crucial subsequent phase for this research endeavor. From our results, we advocate for future investigations focusing on the application of ML to predict thrombolysis resistance to embrace a multi-faceted modeling approach. This strategy will enhance the robustness and reliability of predictive models in this domain.