Keywords

1 Introduction

Recently, time-series data mining has played an important role in various real-life applications, such as healthcare [10], astronomy [3], and aerospace [11]. The availability of big data provides researchers the unprecedented opportunity to deploy high accurate models in real-life applications. The success of deep learning time series models has been validated by their high accuracy. However, the black-box nature makes the decision-making process of high accuracy models less explained [1]. This represents a big barrier for decision makers in trusting the system. For example, in life-changing decisions such as critical disease management, it is important to know the reasons behind the critical decision of choosing a treatment line over another one [9]. An inappropriate treatment decision can cost human lives in addition to a substantial monetary loss [26]. Therefore, it is vital to understand the opaque models’ decision process by either prioritizing interpretability during the model development phase, or developing post-hoc explanation solutions.

To build trust between humans and decision-making systems, EXplainable Artificial Intelligence (XAI) methods are increasingly accepted as effective tools to explain machine learning models’ decision-making processes. XAI methods aim at increasing the interpretability of the models whilst maintaining the high-performance levels. There are two XAI dominant paradigms: intrinsic interpretable and post-hoc explanations for opaque models [29]. Examples of intrinsic interpretable models include linear regression, decision trees, rule sets, etc. On the other hand, examples of post-hoc explanation methods include LIME [19], native guide, and SHAP [23, 24]. In this paper, we focus only on post-hoc XAI methods. A lot of efforts have been made to provide post-hoc XAI for image and vector-represented data while significantly less attention has been paid to time series data [18]. The complex and time-evolving nature of time series makes the explanation models one of the most challenging tasks [5].

On the light of desirable counterfactual method properties, we propose a Motif-Guided Counterfactual Explanation (MG-CF), a novel model that generates interpretable, intuitive post-hoc counterfactual explanations of time series classification predictions that balance between validity, proximity, interpretability, contiguity and efficiency. The contributions are the following:

  1. 1.

    We incorporate motif mining algorithms for guiding the counterfactual search. In particular, we make full use of the interpretability power of motifs to provide interpretable information in the decision-making process. Our method does not require the use of class activation maps to search for the counterfactual explanation, which makes it model-agnostic.

  2. 2.

    We conduct experiments on six real-life datasets from various domains (image, spectro, ECG, and motion) and show the superiority of our methods over state-of-the-art models.

To the best of our knowledge, this is the first effort to leverage the a-priori mined motifs to produce counterfactual explanations for time series classification. The rest of this paper is organized as follows: in Sect. 2, background and related works are described. Section 3 introduces the preliminary concepts. Section 4 describes our proposed method in detail. We present the experimental results and evaluations in comparison to other baselines in Sect. 5. Finally, we conclude our work in Sect. 6.

2 Background and Related Work

In the post-hoc interpretability paradigm, one of the most prominent approaches is to identify important features given a prediction through local approximation, such as LIME [19], LORE [7], TS-MULE [21], and SHAP [14]. In particular, LIME is a feature-based approach that operates by fitting an interpretable linear model to the classifier’s predictions of random data samples, weighted based on their distance to the test sample. LORE is an extension work based on LIME, which is a local black box model-agnostic explanation approach based on logic rules. TS-MULE is also an extension to LIME with novel time series segmentation techniques and replacement methods to enforce a better non-informed values exchange. SHAP is a unified framework that operates by calculating feature importance values using model parameters. In addition, visualizing the decision of a model is also a common technique for explaining model predictions [16]. In the computer vision domain, visualization techniques have been widely applied to different applications successfully, such as highlighting the most important parts of images to class activation maps (CAM) in convolutional neural networks [15]. Schlegel et al. [20] tested the informativeness and robustness of different feature-importance techniques in time series classification. LIME was found to produce poor results because of the large dimensionality by converting time to features; in contrast, saliency-based approaches and SHAP were found to be more robust across different architectures.

An alternative method to feature-based methods has been proposed by Wachter et al. [25], which aims at minimizing a loss function and using adaptive Nelder-Mead optimization to encourage the counterfactual to change the decision outcome and keep the minimum Manhattan distance from the original input instance. Similarly, another method that is used to deal with the plausibility and feasibility issues of the generated counterfactual explanation has been proposed, which is called GeCo. The model achieves the desirable counterfactual properties by introducing a new plausibility-feasibility language (PLAF) [22]. Both GeCo and wCF focus on structured tabular datasets. Since these two methods explore a complete search space, they are not adequate to be used in a high dimensional feature space such as in the case of time-series data.

Recently, an instance-based counterfactual explanation for time series classification has been proposed [5]. The instance-based counterfactual explanation uses the explanation weight vector (from the Class Activation Mapping) and the in-sample counterfactual (NUN) to generate counterfactual explanations for time series classifiers. The proposed technique adapts existing training instances by copying segments from other instances that belong to the desired class. More recently, MAPIC, an interpretable model for TSC based on Matrix Profile, shapelets, and decision tree has been proposed by [6]. Finally, a counterfactual solution for multivariate time series data called CoMTE has been proposed by Etes et al. [2], which focuses on observing the effect of turning off one variable at a time.

3 Preliminary Concepts

Notation. We define a time series \(T = \{t_1, t_2, ..., t_m\}\) as an ordered set of real values, where m is the length, then we can define a time series dataset \(\boldsymbol{T}=\{T_{0},T_{1},...,T_{n}\}\) as a collection of such time series where each time series has mapped to a mutually exclusive set of classes \(C=\{c_1, c_2, ..., c_l\}\) and used to train a time series classifier \({f}: \boldsymbol{T}\xrightarrow {}C\), where \(\boldsymbol{T} \in \mathbb {R}^k\) is the k-dimensional feature space. For each \(T_i\) in time series dataset \(\boldsymbol{T}\) associated with a class \({f}(T_i)=c_{i}\), a counterfactual explanation model \(\mathcal {M}\) generates a perturbed sample \(T'_i\) with the minimal perturbation that leads to \({f}(T'_i)=c'_i\) such that \(c_i \ne c'_i\).

According to [5, 13], a desire counterfactual instance \(x_{cf}\) should obey the following initial properties :

  • Validity: The prediction of the to-be-explained model f on the counterfactual instance \(x_{cf}\) needs to be different from the prediction of the to-be-explained model f on the original instance x (i.e., if \(f(x)=c_i\) and \(f(x_{cf})=c_j\), then \(c_i \ne c_j\)).

  • Proximity: The to-be-explained query needs to be close to the generated counterfactual instance, which means the distance between x and \(x_{cf}\) should be minimal.

  • Sparsity: The perturbation \(\delta \) changing the original instance \(x_0\) into \(x_{cf} = x_0 + \delta \) should be sparse (i.e., \(\delta \approx \epsilon \)).

  • Model-agnosticism: The counterfactual explanation model should produce a solution independent of the classification model f, high-quality counterfactuals without prior knowledge of the gradient values derived from optimization-based classification models should be generated.

  • Contiguity: The counterfactual instance \(x_{cf} = x + \delta \) needs to be perturbed in a single contiguous segment which makes the solution semantically meaningful.

  • Efficiency: The counterfactual instance \(x_{cf}\) needs to be found fast enough to ensure it can be used in a real-life setting.

4 Motif-Guided Counterfactual Explanation (MG-CF)

In this section, we describe our proposed Motif-Guided Counterfactual Explanation method. The general architecture is illustrated in Fig. 1, there are two main steps: motifs mining and counterfactual explanation generation.

Fig. 1.
figure 1

Motif-Guided Counterfactual Explanation

Full size image

4.1 Motif Mining

Motifs, also known as shapelets, are time series sub-sequences that are maximally representative of a class [27]. Motifs provide interpretable information beneficial for domain experts to better understand their data. To mine the most prominent motifs, we apply the Shapelet Transform (ST) algorithm proposed by [12]. To generate the most prominent motifs, a set of candidate motifs need to be generated, a distance measure between a motif and each time series instance should be defined, and a measure of the discriminatory power of the mined motifs [12] need to be defined. The details of the algorithm is shown in Algorithm 1. The algorithm starts by sampling specific-length intervals from the time series instance to generate candidate motifs (line 7). The motifs’ discriminatory power is then estimated by first computing their distance to all the instances from each class (line 9), and then computing the distance correlation to the target class (line 10). Finally, the motifs are sorted and only the best motif from each class is retained (lines 14, 16). Figure 2a and 2b show the two extracted motifs of different classes from ECG 200 dataset.

Candidates Generation. Given a time series T with length m, a subsequence S of length l of time series T is a contiguous sequence of l points in T, where l \(\le \) m. For a given m-lengthed time series, there are \((m-l) + 1\) distinct subsequences of length l. The set of all subsequences of length l for a dataset is described as:

$$\begin{aligned} W_l = \{W_{1,l},...,W_{n,l}\}, \end{aligned}$$
(1)

where \(W_{i,l}\) is defined as the set of all subsequences of length l for time series \(T_i\). To speed up the process of finding motifs, instead of considering all the lengths that range in (3, m), similar to [12], we only consider 0.3, 0.5 and 0.7 percent length of the whole time series for each dataset. In this case, the set of all candidate motifs for data set T is:

$$\begin{aligned} W = \{W_{0.3m}, W_{0.5m},W_{0.7m}\}, \end{aligned}$$
(2)

We denote the candidates generation as generateCandidates in Algorithm 1.

Measuring Distances. The distance between two subsequences S and R of length l is defined as

$$\begin{aligned} dist(S, R) = \sum _{i=1}^{l}(s_i - r_i)^2 \end{aligned}$$
(3)

To calculate all distances between a candidate motif S and all series in T, we denote \(d_{i,s}\) as the distance between a subsequence S of l and time series \(T_i\), then

$$\begin{aligned} d_{i,S} = \mathop {min}_{R \in W_{i,l}}dist(S,R) \end{aligned}$$
(4)
$$\begin{aligned} D_S = <d_{1,S}, d_{2,S}, ..., d_{n,S}> \end{aligned}$$
(5)

We denote the process of measuring distance as findDistances in Algorithm 1.

Measuring the Discriminatory Power of a Motif. Information gain has been used to evaluate the quality of a motif in the previous shapelet papers [17, 27, 28]. Firstly, the distance list \(D_S\), defined in Eq. 5, is sorted, then the information gain is evaluated on the class values for each possible split value. Ye et al. propose to use an early abandon technique on calculating \(D_S\) through maintaining an upper bound on the quality of the candidate whilst generating \(D_S\) [28]. If the upper bound falls below the best found so far, the calculation of \(D_S\) can be abandoned. After each \(d_{i,S}\) is found, the current best information gain split is calculated and the upper bound is found, assuming the most optimistic division of the remaining distances. We denote the process of measuring the discriminatory power of a motif as assessCandidate in Algorithm 1.

figure a

4.2 CF Generation

After motifs are extracted from the training dataset, the next step is to generate CF explanations. In this section, we introduce the counterfactual explanation generation process as outlined at Algorithm 2. The CF generation starts by using the pre-trained model f to predict the classification result of the test dataset \(\textit{samples}\) (line 1). We set the target classes of the test dataset as the opposite label of the prediction results (line 2). To generate an explanation of a query time series X, we consider the motif’s existing segment from the target class as the most prominent segment (lines 4–7). Therefore, we replace the motif existing part with the target class motif and preserve the remaining parts (line 9). In this case, the counterfactual explanation for the input query time series is generated (line 10). It is important to note that MG-CF does not require the use of class activation maps which makes it a model-agnostic model. Figure 2 shows examples of generated counterfactual explanations for the ECG 200 dataset of the Myocardial Infarction and Normal Heartbeat class.

figure b
Fig. 2.
figure 2

Example MG-CF explanations for the ECG200 dataset

Full size image

5 Experimental Evaluation

In this section, we outline our experimental design and discuss our findings. We conduct our experiments on the publicly-available univariate time series data sets from the University of California at Riverside (UCR) Time Series Classification Archive [4]. We test our model on five real-life datasets from various domains (image, spectro, ECG, and motion). The number of labels of each dataset is binary and the time series classes are evenly sampled across the training and testing partitions. The length of the time series in the data sets varies between 96 and 512. Table 1 shows all the dataset details (number of classes, time series length, training size, testing size, and the type of time series).

5.1 Baseline Methods

We evaluate our proposed SG-CF model with two other baselines, namely, Alibi [8] and Native guide counterfactual explainers [5].

  • Alibi Counterfactual (Alibi): The Alibi generates counterfactuals by optimizing a new objective function that includes class prototypes generated from encoders [8]. The method leverages the resulting autoencoder reconstruction error after decoding the counterfactual instance \(x_{cf}\) as a measure of fitness to the training data distribution. The additional loss term \(L_{AU}\) penalizes out-of-distribution counterfactual instances.

  • Native guide counterfactual (NG-CF): NG-CF is the latest time series counterfactual method that perturbs the query time series by inserting segments from the opposing class [5]. Although the method is model agnostic, the method reaches its full potential when class activation weights are available. NG-CF uses Dynamic Barycenter (DBA) averaging of the query time series x and the nearest unlike neighbor from another class to generate the counterfactual example.

The source code of our model, competing baselines, and the experimental dataset are available on the MG-CF project websiteFootnote 1.

5.2 Experimental Results

The goal of our experiments is to assess the performance of the baseline methods with respect to all the desired properties of an ideal counterfactual method. To evaluate our proposed method, we compare our method with the other two baselines in terms of several metrics: L1 distance (proximity), sparsity level, number of independent segments is perturbing (contiguity), the label flip rate (validity), and the run time (efficiency).

L1 distance, which is defined in Eq. 6, measures the distance between the counterfactuals and the original samples, a smaller L1 distance is desired.

$$\begin{aligned} proximity = \Vert x_{cf} - x\Vert \end{aligned}$$
(6)
Table 1. UCR datasets metadata (each dataset is binary labeled)
Full size table

Sparsity level, which indicates the level of time series perturbations. A high sparsity level that is approaching \(100\%\) is desirable, which means the time series perturbations made in x to achieve \(x_{cf}\) is minimal. We computed the sparsity level using a new metric that we formulate in Eqs. 7 and 8.

$$\begin{aligned} sparsity = 1- \frac{\sum _{i=0}^{len(x)}g(x_{cf_i}, x_i)}{len(x)} \end{aligned}$$
(7)
$$\begin{aligned} g(x, y)=\left\{ \begin{array}{ll} 1, &{} \text{ if } x \ne y \\ 0, &{} \text{ otherwise } \end{array}\right. \end{aligned}$$
(8)

The number of independent non-contiguous segments is also investigated to show the contiguity, which is shown in Fig 3. The lower the number of independent non-contiguous segments the better. Finally, we define the validity metric by comparing the label flip rate for the prediction of the counterfactual explanation result, we computed the flip rate following the formula of Eq. 9.

$$\begin{aligned} flip\_rate = \frac{num\_flipped}{num\_testsample}, \end{aligned}$$
(9)

where we denote the num_flipped as the number of generated counterfactual explanation samples that flip the label, and the num_testsample is the total number of samples in the test dataset that we used to generate counterfactual explanation samples. The closer the label flip rate is to 1, the better.

Figure 3 and 4 show the evaluation results on the CF desired properties assessed using the aforementioned metrics. Since our proposed method relies on an instance-based counterfactual explanation, which generates a CF for each time series in the dataset, we calculate the mean value and the standard deviation for all CF samples for each dataset to show the overall performances. We note that Alibi achieves the lowest L1 distance (proximity) in the four datasets, which is highly desirable. However, Alibi minimizes proximity in the cost of sparsity level, contiguity (number of independent segments), and efficiency (running time). Alibi achieves the lowest sparsity level and run time, which entails that the method generates CF explanations with less proximity, contiguity, and efficiency.

Fig. 3.
figure 3

The sparsity level (the higher the better) and number of independent segments (the lower the better) of the CF explanations

Full size image
Fig. 4.
figure 4

The average running time (the lower the better), average L1 distance (the lower the better), and the label flip rate (the higher the better) of the CF explanations

Full size image

We also note that NG-CF results in highly valid counterfactual explanations that are guaranteed to have the desired class. Since the method does not rely on any optimization function to generate new shapes, it simply copies fragments from existing training time series instances that the prediction model f has already learned. Therefore, it is expected that the prediction model f recognizes the copied segments with high confidence. Although highly valid, NG-CF sacrifices the other metrics (sparsity, contiguity) for maximizing validity. Figures 3 and 4 show that our proposed MG-CF results in a higher sparsity level, a lower number of independent perturbed segments (contiguity), and a lower running time (efficiency) compared to NG-CF and Alibi. Finally, although Alibi and NG-CF can lead to high proximity and validity counterfactuals, they are not valid and sparse respectively. Thus, MG-CF shows a good balance across all the metrics without maximizing one property and compromising the other.

6 Conclusion

In this paper, we propose a novel model that derives intuitive, interpretable post-hoc counterfactual explanations of time series classification models that finds balance between validity, proximity, sparsity, contiguity, and efficiency desirable properties. The main limitation of current time series explainers is that they produce perturbations that maximize a metric on the cost of others. This shortcoming limits the interpretability of their resulting counterfactual explanations. We address these challenges by proposing MG-CF, a motif-based model that produces high-quality counterfactual explanations that are contiguous and sparse. The MG-CF method guides the perturbations on the query time series resulting in significantly sparse and more contiguous explanations than other state-of-the-art methods. This is the first effort to leverage the a-priori mined motifs to produce high-quality counterfactual explanations. There are spaces for extensions of our work with time series datasets with high dimensionality and multiple-classes case. As a future direction of this work, we would like to extend our work to multiple-class case time series dataset and also focus on generating the counterfactual explanations for high dimensional time series data.