Introduction

Advances in drug development and in treatment strategies have dramatically improved the prognosis and life quality of patients with rheumatoid arthritis (RA) [1, 2]. Further progress is still necessary, because the patients display important variability in the response to all the disease-modifying anti-rheumatic drugs (DMARD), including response to methotrexate (MTX), which is often used in monotherapy as the first DMARD immediately after RA diagnosis. This inter-individual variability means that a significant fraction of patients on MTX fail to sufficiently improve. Ideally, the non-responder (NR) patients should be prescribed a different DMARD or a DMARD combination as the first treatment in place of MTX. This goal will require predicting MTX non-response [3, 4]. A promising approach could be the previously published clinical pharmacogenetics model of response to MTX (CP-MTX) [5]. This model combines clinical and pharmacogenetics data (sex, DAS at baseline, RF and smoking and four SNPs in genes of the folate pathway) in a scoring system that has been validated in three groups of patients [5, 6]. Nevertheless, CP-MTX has not attracted research beyond its developers. This lack of interest could be due to uncertainty on its clinical utility. In effect, performance of CP-MTX has been reported with statistical parameters pertaining to three strata of the score, but this information is difficult to translate in value for guiding drug prescription. For example, the discovery study reported 95% positive predictive value (PPV) for response (scores ≤ 3.5), 86% negative predictive value (NPV) for non-response (scores > 6), and 40% of the patients were left without any prediction (scores > 3.5 ≤ 6). Although the PPV and NPV results seem worthy, their complexity hampers understanding its clinical utility. In addition, subsequent replication sets of RA patients have shown a decrease in the three parameters (70%, 72%, and 32%, respectively, in the first replication set [5]; and 47%, 81%, and 25%, respectively, in the second set [6]) making it even harder to assess the utility of the model. The decrease in PPV and NPV is indicative of less utility, but this negative aspect could be counterbalanced by the reduction in the percentage of patients without any prediction. An additional reason for the lack of interest in CP-MCT could be its reliance on pharmacogenetics given the low reproducibility and predictive power that MTX pharmacogenetics has demonstrated until now [7,8,9]. Consequently, our objective has been to assess the potential impact in clinical management of implementing CP-MXT for prescribing RA treatment. In addition, we have aimed to evaluate the reproducibility of its performance and the role of its pharmacogenetics component. Accordingly, we have focused our analysis on the clinical benefit-cost expected from CP-MTX implementation, we have analyzed a larger number of RA patients than previous studies, and we have addressed the contribution of the pharmacogenetics component.

Materials and methods

Patients and clinical component of the model

For this study, we selected 720 RA patients with complete information for CP-MTX that have been recruited with written informed consent and the approval of the local ethics committees. The study was approved by the Comite Etico de Investigacion Clinica de Galicia (Santiago de Compostela, Spain). The patients were of European Caucasian ancestry and more specifically of Spanish (221), Portuguese (229), and Polish (270) origin. All have received MTX as the only DMARD for 6 months. Nevertheless, 23.9% have been treated previously with other DMARD. In addition, most patients combined MTX with corticosteroids (72.8%) or non-steroidal anti-inflammatory drugs (72.0%). These and other clinical characteristics, including baseline DAS28, sex, RF and smoking status for CP-MTX are presented in Table 1.

Table 1 Clinical characteristics of the 720 RA patients included in the study and their stratification by response to MTX at 6 months of treatment
Full size table

Genetic component of the model

The same four SNPs previously reported in the published CP-MTX prediction model were tested here (MTHFD1 rs2236225, AMPD1 rs17602729, ITPA rs1127354 and ATIC rs2372536) [5]. The corresponding genotypes were determined by PCR amplification followed by single-base extension (primers and probes available upon request) with the SNaPshot Multiplex Kit (Applied Biosystems, Foster City, California). The complete genotypes at the four SNPs were required for all samples. Duplicate genotypes of 10% of the samples, concordance (all p > 0.05) with the Hardy–Weinberg equilibrium and with SNP frequencies in the HapMap European collection were used for quality control [10].

Evaluation of model predictions

CP-MTX was applied as described [5], except for the use of DAS28 in place of DAS with equivalences calculated as DAS28 = 1.072 × DAS + 0.938 [11]. Our focus on clinical relevance of the model limited the analysis to a 2 × 2 contingency table (Table 2), where emphasis was put only in predicting patients requiring an alternative treatment than the received. That is, the non-responder patients (NRobs) that did not reach low disease activity (LDA) at 6 months on MTX monotherapy. The accuracy of this prediction was characterized by its positive predictive value (PPV = D/(C+D)) and false negative rate (FNR = B/(B+D)). In addition, we performed a benefit-cost analysis to facilitate understanding of the clinical utility of CP-MTX. In this analysis, the expected benefit is equal to the correctly predicted NR (D in Table 2), as they will be prescribed an alternative treatment. Whereas the expected cost is the fraction of wrongly predicted NR (C in Table 2), as they will receive unnecessary treatment. In addition, baseline patient variables were compared between NRobs and patients achieving LDA by using the Chi-square test or logistic regression. Also, receiver operating characteristic (ROC) analysis was used to determine cut-offs for the model based only in the clinical component of the CP-MTX model. All analyses were performed using SPSS version 15.0 (SPSS Inc., Chicago, IL).

Table 2 Contingency table showing the definitions used for evaluation of CP-MTX performance
Full size table

Results

The patients with RA (n = 720, Table 1) included in this study were treated with MTX monotherapy, although most of them received also corticosteroids (72.8%) or NSAID (72.0%). Most were DMARD naive, although 23.9% have previously received another DMARD. Exactly two thirds of the patients (n = 480) did not reach LDA (DAS28 < 3.2) at 6 months representing the NRobs group, which should be predicted by CP-MTX to indicate an alternative treatment at baseline. The most notable differences between patients achieving LDA and NRobs were that the latter showed higher DAS28 and bone erosions at baseline and were less frequently DMARD naive (Table 1). The NRobs also were more frequently women and younger and during the treatment received a higher maximum dose of MTX. Two of these variables are included in CP-MTX (DAS28 and sex), but the other two variables in the model, RF and smoking were not significantly different between NRobs and LDAobs patients.

Application of the CP-MTX scoring to clinical variables amounted to a mean of 3.2 (range 0.0–6.5) (Table 3). In turn, the total pharmacogenetics component was smaller (mean 1.8) and showed less variability (range 1.0–4.0) than the clinical component. Jointly, the mean total CP-MTX score was 5.0 (range 1.0–10.5), with 41.8% of the patients showing values corresponding to NRpred. As expected, the clinical characteristics corresponding to high scores were more common in NRobs patients than in LDAobs (Table 3). The largest differences were observed with baseline DAS28. In contrast, none of the four genotypes showed a significant difference (Table 3). The CP-MTX mean total scores reflected these results with values of 4.3 and 5.4 for LDAobs and NRobs patients, respectively (p = 5.1 × 10−13).

Table 3 Distribution of RA patients according to CP-MTX scoring and prediction
Full size table

Performance of CP-MTX for identifying NRobs was defined by 79.7% PPV (Table 4), but also by 50.0% FNR that complicates interpretation. The two parameters fell in the range observed in previous studies (Table 4), whose implications were also hard to extract. In contrast, the benefit-cost analysis was more revealing (Table 4). It showed that 33.3% of our patients will benefit from the CP-MTX prediction, as this is the fraction that will be correctly predicted as NR and will be given an alternative treatment. The cost of the CP-MTX prediction will be of overtreating 8.5% of the RA patients, which will respond to MTX but wrongly be classified as NRpred. This benefit-cost relation seems favorable to the use of CP-MTX, although a significant fraction of NRobs patients will be missed by the predictions (33.3% of the total). We also calculated the benefit-cost values corresponding to the three patient sets analyzed in previous CP-MTX studies (Table 4) [5, 6]. Surprisingly, these values were very similar: all showed a benefit of about 33% at a cost of about 8%. This could be expressed as reducing 4 NR patients at the cost of 1 patient receiving alternative treatment that was unnecessary, a result showing potential to improve RA management.

Table 4 Performance of CP-MTX for prediction of NRobs in our patients and in previous studies
Full size table

Analysis of important strata in our patients (Table 5) showed that the DMARD-experienced patients were more frequently NRobs than the DMARD-naive patients (76.7 vs. 63.4%, p = 0.001). In addition, CP-MTX performance was significantly better in the experienced than in the naive patients, with higher PPV and lower expected cost (Table 5). However, CP-MTX retained a favorable benefit-cost relationship in DMARD-naive patients (Expected benefit = 32.5% and expected cost = 10.4%). Other important stratification was determined by the use of concomitant corticosteroids, but, in this case, no differences were observed (Table 5).

Table 5 Performance of CP-MTX for prediction of NRobs in important patient strata
Full size table

As signaled above, the pharmacogenetics component of CP-MTX showed a lower contribution to the scores than the clinical part, and none of the 4 SNP was different between NRobs and LDAobs (Table 3). Therefore, we assessed the value of implementing a model including only the clinical component. As a first step, a new cut-off was determined by ROC analysis (Supplementary Table 1). Applying the resulting 3.5 threshold, the clinical model showed a somehow better performance than CP-MTX for prediction of NRobs (PPV = 81.7%, FNR = 38.6%, Supplementary Table 2), and for the benefit-cost analysis (Expected benefit = 41.0% and expected cost = 9.2%). Notably, only sex and disease activity at baseline contributed significantly to this prediction (not shown)

Discussion

Our results have clearly shown the potential value of CP-MTX for improving management of RA patients with an expected benefit of 33% at a cost of 8%. This potential was manifest because of our new focus, placed in reducing the fraction of NR patients, and of the explicit definition of benefits and costs expected from CP-MTX. Confidence in the results is supported by the large number of patients and by demonstrating very similar benefit-cost performance in our reanalysis of previous studies [5, 6].

An important aspect of our work is the clarity introduced in the analysis by focusing on the most relevant clinical problem: reduction of the fraction of NR patients [1, 2]. Another remarkable aspect of our results is the increased reproducibility of the critical benefit-cost values. This consistency of results has been obtained in patients that were diverse: from a clinical trial [5], or from conventional practice (here and in [6]); DMARD-naive or DMARD-experienced; and with or without concomitant corticotherapy. Therefore, the robust reproducibility of CP-MTX shows promise for broad validity.

The previous CP-MTX studies have already shown the low weight of the pharmacogenetics component [5, 6]. However, it was still considered necessary based on improved performance parameters [5, 6]. In contrast, we have found a negligible contribution of pharmacogenetics in our patients, both with our benefit-cost analysis and with the parameters previously reported (Supplementary Table 2). This result is important because the need to obtain the genetic information prior to start treatment is a critical barrier for the implementation of CP-MTX. However, we expect that progress in pharmacogenetics will permit to improve the model in the future [8, 9].

A limitation of our study is the requirement of treatment continuation for 6 months, which have excluded patients with earlier discontinuation. However, it should be noted that most patients (70–90%) on MTX monotherapy continue it until 6 months [12,13,14,15,16,17], and that early drug withdrawals are unfrequently (4–25%) attributable to lack of efficacy [12, 13, 15, 16]. In addition, the benefit achievable with CP-MTX is limited to change to alternative treatments. Unfortunately, it will not translate completely in good responder patients.

A comment is required on the place that CP-MTX could have in the context of treatment recommendations [2, 18,19,20,21]. In some countries, health authorities demand that all RA patients are initially treated with a combination of MTX plus a second DMARD unless contraindicated. This view is reflected in some treatment guidelines that avoid MTX monotherapy [22, 23]. However, the most recent and widely accepted recommendations, from the EULAR, ACR and other rheumatologist associations, endorse MTX as monotherapy for new RA patients [2, 20, 21]. This advice is strong for patients with low disease activity and conditional for patients with higher disease activity in some of these guidelines [20, 21], whereas in the EULAR 2016 recommendations combination therapy is no longer explicitly mentioned as initial treatment [2]. Currently, none of the guidelines includes predictions of treatment outcome as a tool for drug choice. They only enumerate factors associated with a poor prognosis that should be considered for selecting a more aggressive therapy. The list of poor prognosis factors includes some of the comprised in the clinical component of CP-MTX (high disease activity, smoking, presence of RF), but without further specification. Therefore, the CP-MTX could fill the role of prediction model in patients in which MTX monotherapy is considered.

In conclusion, our results indicate CP-MTX is able to predict patients that will fail to respond if prescribed MTX monotherapy. The expected benefit from implementing CP-MTX in clinical management is a decrease in the fraction of NR patients of about 33%. These patients will be changed from MTX monotherapy to more aggressive treatments. A change in prescription that will be justified in three of each four patients. The CP-MTX could be further improved by including other clinical variables, as presence of erosions or previous experience with DMARDs, and modifying the pharmacogenetics component to update the list of included SNPs.