1 Introduction

Nowadays, a wide range of real-life applications such as computer vision [1, 2], speech processing, natural language understanding [3], health [4], and military fields [5, 6] make use of Machine Learning (ML) models for decision making or prediction/classification purpose. However, those models are often implemented as black boxes which make their predictions difficult to understand for humans. This nature of ML-models limits their adoption and practical applicability in many real world domains and affect the human trust in them. Making ML-models more explainable and transparent is currently a trending topic in data science and artificial intelligence fields which attracts the interest of several researchers.

Explainable AI (XAI) refers to the tools, methods, and techniques that can be used to make the behavior and predictions of ML models to be understandable to human [7]. Thus, the higher the interpretability/explainability of a ML model, the easier it is for someone to comprehend why certain decisions or predictions have been made.

Multiple interpretability approaches are based on additive models where the prediction is a sum of individual marginal effects like feature contribution [8], where a value (denoting the influence on the output) is assigned to each feature. One of the latest proposed methods is based on mathematical Shapeley Values and was introduced by Scott et al. [9] as SHAP (for SHapley Additive exPlanations). It relies on combining ideas from cooperative game theory and local explanations [10]. LIME (Local Interpretable Model-agnostic Explanations), introduced by Ribeiro et al. [11], is also one of the most famous local explainable models. It explains individual predictions of any classifier or regressor in a faithful and intelligible way, by approximating them locally with an interpretable model (e.g., linear models, decision trees). However, having a global explanation of the model can be challenging as it is more complicated to maintain a good fidelity - interpretability trade off. To this end, authors in [11] proposed an approach, called submodular Pick which is an algorithm aiming to maximize a coverage function of total feature importance for a set of instances. While maximizing the coverage function is NP-Hard, authors make use of a greedy algorithm which adds iteratively instances with the highest marginal coverage to the solution set, offering a constant-factor approximation to the optimum. The selected set is the most representative, non-redundant individual explanations of the model.

In this paper, our aim is to introduce a new approach to select individual instances (explanations) to be considered for global explanation to ensure that the picked group reflects the global behavior of the black-box model. Unlike submodular optimization proposed in [11], we advocate to consider the problem of picking representative instances as a co-selection task. The idea is to apply a similarity preserving co-selection approach to select a set of instances and features on the explanations provided by any explainer. In fact, feature or instance selection has been widely considered separately in the literature to remove noise, irrelevant and redundant features or instances in datasets [12,13,14,15,16]. Unfortunately, selecting features and instances separately and sequentially is time consuming, especially when dealing with large scale datasets. To overcome this problem, co-selection or the simultaneous selection of features and instances is proposed making use of the duality between feature space and instance space. In this context, several approaches have been proposed. For instance, Kuncheva et al. [17] proposed a genetic algorithm that simultaneously select features and reference cases to improve the performance of nearest neighbor classifiers. Derrac et al. [18] suggested an evolutionary model based on cooperative coevolution to perform co-selection in nearest neighbor classification. GarcíA-Pedrajas et al. [19] proposed a scalable, almost any size, method for concurrent instance and feature selection. In another side, similarity preserving approaches have been considered in the literature with the aim of evaluating features by their ability to preserve locality. For instance, Zhao et al. [20] introduced a similarity preserving feature selection framework that overcomes common weakness in handling feature redundancy. Ma et al. [21] proposed a similarity preserving method that generate unseen visual features from random noises concatenated with semantic descriptions. Shang et al. [22] suggested UFSRL, a framework that used local similarity preserving for feature selection.

1.1 Contributions

The technical contributions of this paper are summarized as follows.

  • We propose a new approach, called CoSP, for a global explainability of black box machine learning.

  • The proposed approach selects individual explanations to provide global explanation for machine learning models.

  • CoSP is based on a similarity preserving co-selection approach.

  • Experiments are conducted to validate the efficacy of these contributions.

  • We release a performant implementation of CoSP at [23].

The original version of this work is published at WISE’22 [24]. The main changes in this paper are presented below:

  • Creating a new section (Related Work) that presents some modern explainability methods proposed in the literature to further clarify the importance of interpretability of machine learning models.

  • Detailing, in the Proposed approach section, the alternative optimization procedure applied to the objective function.

  • Adding computational complexity in the Algorithm Analysis subsection.

  • Conducting further experiments to validate the effectiveness of CoSP by comparing it against four approaches including, Random, Greedy [25], Parzen [26] and LIME [11], combined with Submodular Pick (SP) and Random Pick (RP).

The paper is structured as follows. Section 2 introduces the related work. Section 3 provides a necessary background on LIME method. In Section 4, we present our approach allowing for a global explanation of black box ML models. Section 5 shows the preliminary experiments done to validate our proposal. In Section 6, we conclude the paper and draw some research lines for future work.

2 Related work

Interpretability of ML models reflects the ability to provide meaning in understandable terms to human. It is crucial to trust the system and get insights based on its decisions. Quality of an explanation could be improved by making it more Interpretable, Faithful, and model-agnostic [27]. Faithfulness represents how the explanation is describing the reality of the model. Model-agnostic methods are used for any type of model. Several explainability methods are proposed in the literature. LIME introduced by Ribeiro et al. [11], is one of the well-known examples. It is a framework which explains a prediction by approximating it locally using an interpretable model. Other methods were proposed later, for instance, Burkart et al. [28] provided a survey that presents the main explainable of supervised machine learning methods. Lundberg et al. [29] suggested a novel explanation that improves the interpretability of tree-based models by directly measuring local feature interaction effects. Vlahek et al. [30] introduced an iterative approach to learning explainable features, where new features are generated with each iteration and high quality dissimilars are selected. Dinh et al. [31] suggested a consistent feature selection for analytic deep neural networks. Cancela et al. [32] proposed E2E-FS, a feature selection algorithm providing both precision and explainability in a smart way. Wang et al. [33] proposed RC-Explainer, a Reinforced Causal Explainer for Graph Neural Networks. A poweful framework that generate faithful and concise explanations to unseen graphs. Moritz et al. [34] introduced CoDA Nets, a powerful classifiers with a high degree of inherent interpretability. Table 1 gives a concise overview of other existing explainability algorithms.

Table 1 Overview of explainability methods
Full size table

3 Background on LIME

The basic idea of LIME is to replace a data instance x by its interpretable representations \(x'\) thanks to a mapping function \(\Phi (x)\). For example, an image will be represented as a group of super-pixels, a text as binary vectors indicating the presence or the absence of a word. The interpretable representations are more easily understandable and close to human intuition. Then, \(x'\) is perturbed to generate a set of new instances. The black box model is used to make predictions of generated instances from \(x'\) which are weighted according to their dissimilarity with \(x'\). Now, for the explanation purpose, an interpretable model, such as linear models, is trained on weighted data to explain prediction locally (see, Algorithm 1).

Algorithm 1
figure a

Sparse linear explanations LIME.

Full size image

3.1 LIME: fidelity-interpretability trade-off

Authors in [11] define an explanation as a model \( g \in G\), where G is a class of potentially interpretable models (e.g., linear models, decision trees). Let \( \Omega {(g)} \) be a measure of complexity (as opposed to interpretability) of the explanation g. For example, for linear models \(\Omega (g)\) may be the number of non-zero weights. The model being explained is denoted by \(f : \mathbb R^d \longrightarrow \mathbb R\). Let now \(\pi _x\) defines a locality around x and \(\mathcal {L}(f; g; x) \) be a measure of how unfaithful g is in approximating f in the locality \(\pi _x\). The explanation produced by LIME is then obtained by the following minimization problem [11]:

$$\begin{aligned} \xi {(x)} = \mathop {\textrm{argmin}}\limits _{g\in G} \mathcal {L}(f;g;\pi _x) +\Omega {(g )} \end{aligned}$$
(1)

3.2 Explaining global behavior

LIME explains a single prediction locally. Then, it picks K explanations which must be representative to show to the user. The Submodular Pick is used to choose instances to be inspected for global understanding. The quality of selected instances is critical to get insights from the model in a reasonable time (see, Algorithm 2). Let \(\mathrm {\textbf{X}}\) (with \(|\mathrm {\textbf{X}}|= n\)) be the set of instances to explain, Algorithm 2 calculates \(\mathrm {\textbf{W}}\in \mathbb R^{n \times d'}\) an explanation matrix using each individual explanation given by Algorithm 1. Then, it computes \( (I_{j})\) global feature importance for each column j in W, such that the highest importance score is given to the feature explaining an important number of different instances. Submodular Pick aims then at finding the set of instances V, \(|V| < \mathrm {\textbf{B}}\) that scores the highest coverage, defined as the function which calculates total importance of features in at least one instance. Finally, greedy algorithm is used to build V by adding the instance with highest marginal coverage gain.

Algorithm 2
figure b

Submodular Pick (SP).

Full size image

4 Proposed approach

Fig. 1
figure 1

The proposed framework for a global explanation using a co-selection of features and instances

Full size image

The approach we propose in this paper consists of two sequential phases (see Figure 1). The first is to use LIME (without loss of generality, any other explainer can be used) to obtain the explanations of the predictions for the test data. While the second phase focuses on global explainability by co-selecting the most important test instances and features. Thus, we provide a global understanding of the black-box model.

4.1 Notation

Table 2 summarizes the significant notations used in this paper. Let \({{\,\mathrm{\textbf{E}}\,}}\) be an explanation matrix of n instances and m features. The \(l_{2,1}\)-norm of \({{\,\mathrm{\textbf{E}}\,}}\) is:

$$\begin{aligned} \parallel {{\,\mathrm{\textbf{E}}\,}}\parallel _{2,1} = \sum _{i=1}^{m}\parallel {{\,\mathrm{\textbf{E}}\,}}_{i} \parallel _{2} = \sum _{i=1}^{m} \sqrt{\sum _{j=1}^{n}{{\,\mathrm{\textbf{E}}\,}}_{ij}^{2}} \end{aligned}$$
(2)

and its Frobenius norm (\(l_{2,2}\)) is:

$$\begin{aligned} \parallel {{\,\mathrm{\textbf{E}}\,}}\parallel _{F} =\left( \sum _{i=1}^{m}\parallel {{\,\mathrm{\textbf{E}}\,}}_{i} \parallel _{2}^{2}\right) = \left( \sum _{i=1}^{m} \left( \sum _{j=1}^{n}{} {\textbf {E}}_{ij}^{2}\right) \right) _{}^{1/2} \end{aligned}$$
(3)
Table 2 Summary of symbols and notations
Full size table

4.2 Explanation space

Let f be a black box model, and \(\mathrm {\textbf{X}}\) a test dataset of n instances and \(\Phi (\mathrm {\textbf{X}})=\mathrm {\mathbf {X'}}\) its interpretable representation in \(\mathbb R^p\). First, to obtain an individual explanation of the prediction made by f for each instance \(x_i\) we use LIME by fitting a linear model on a generated dataset around \({x'}_{i}\), the interpretable representation of \(x_i\) . Thus, for each instance \(x_i\), we obtain an explanation of length k (\(k < p\)). It is worthy to note that the length is a parameter set by the user and corresponds to the number of features retained. Once the individual explanations have been obtained, we construct an explanation space represented by \({{\,\mathrm{\textbf{E}}\,}}\in \mathbb R^{n\times m}\), where the dimension m of the explanations space corresponds to the union of the k features of each explanation. We illustrate this step with the following example:

Example

Let \(\mathrm {\mathbf {X'}}\) be the interpretable representation of 3 instances in \(\mathbb R^{500}\), and \(k=3\) be the length of the explanation desired for these three instances. By performing LIME algorithm on \(\mathrm {\mathbf {X'}}\), we obtain 3 explanations of length 3:

$$\begin{aligned} e_i={\left\{ \begin{array}{ll} e_1=\{(f_1,0.5),(f_{25},0.9),(f_4,0.1)\}\\ e_2=\{(f_{17},0.2),(f_6,0.3),(f_{78},0.4)\}\\ e_3=\{(f_{500},0.8),(f_{25},0.7),(f_{1},0.25)\} \end{array}\right. } \end{aligned}$$
(4)

where \(e_1,e_2\), and \(e_3\) are the explanations of \(x'_1\), \(x'_2\) and \(x'_3\) respectively. Thus, the matrix \({{\,\mathrm{\textbf{E}}\,}}\in \mathbb R^{3\times 7}\) can be seen as the concatenation of all the explanations and the union of the set of features obtained by each explanation. Note that the dimension m here is equal to 7.

4.3 Global explicability by co-selection

Understanding the model’s decisions globally remains a complex task. In fact, some approaches like LIME were extended to face this complexity by only picking a group of individual explanations. In this paper, we advocate a method allowing global explainability by co-selecting the most important instances and features over the explanations provided by any explainer. The idea is to find a residual matrix \(\mathrm {\textbf{R}}\) and a transformation matrix \(\mathrm {\textbf{W}}\), which transforms high-dimensional explanations data \({{\,\mathrm{\textbf{E}}\,}}\) to low dimensional data \({{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}\), to maximize the global similarity between \({{\,\mathrm{\textbf{E}}\,}}\) and \({{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}\). After the optimal \(\mathrm {\textbf{W}}\) and \(\mathrm {\textbf{R}}\) have been obtained, the original features and instances are ranked, based on the \(\ell _{2,1}\)-norm values of the rows of \(\mathrm {\textbf{R}}\) and \(\mathrm {\textbf{W}}\), and the top features and instance are selected accordingly.

4.4 Co-selection pick (CoSP)

To perform a co-selection of instances and features on the explanations matrix, we must minimize the following problem as pointed out in [49]:

$$\begin{aligned} \min _{\mathrm {\textbf{W}},\mathrm {\textbf{R}}}{\parallel {{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}-\mathrm {\textbf{R}}^T-{\textbf {Z}}\parallel ^{2}_{F} + \lambda \parallel \mathrm {\textbf{W}}\parallel _{2,1}+\beta \parallel \mathrm {\textbf{R}}\parallel _{2,1}} \end{aligned}$$
(5)

Where:

  • Z is the eigen-decomposition of the pairwise similarity matrix, \(\mathrm {\textbf{A}}\), computed over the explanation matrix \({{\,\mathrm{\textbf{E}}\,}}\). Note that the similarity matrix \(\mathrm {\textbf{A}}\) can be calculated in supervised fashion (e.g. adjacency matrix, fully binary matrix) if the labels of test instances are available, or in unsupervised mode as follows:

    $$\begin{aligned} \mathrm {\textbf{A}}_{ij} = e^{-\frac{\Vert e_i-e_j \Vert ^2}{2\delta ^2}} \end{aligned}$$
    (6)

  • \(\mathrm {\textbf{R}}={\textbf {W}}^T{{\,\mathrm{\textbf{E}}\,}}^T-{\textbf {Z}}^T-\Theta .\), is a residual matrix and \(\Theta \) is a random matrix, usually assumed to be multi-dimensional normal distribution [50]. Note that the matrix \(\mathrm {\textbf{R}}\) is a good indicator of outliers and less important and irrelevant instances in a dataset according to [51, 52].

  • \(\lambda \) and \(\beta \) are regularization parameters, used to control the sparsity of \(\mathrm {\textbf{W}}\) and \(\mathrm {\textbf{R}}\) respectively; and \(\delta \) is a parameter for the RBF kernel used to compute the matrix \(\mathrm {\textbf{A}}\) in the unsupervised mode in (6).

The first term of the objective in (5) exploits the \({{\,\mathrm{\textbf{E}}\,}}\) structure by preserving the pairwise explanations similarity while the second and third terms are used to perform feature selection and instance selection, respectively.

4.4.1 Optimization

In order to minimize (5), we adopt an alternating optimization over \(\mathrm {\textbf{W}}\) and \(\mathrm {\textbf{R}}\) as in[49], by solving two reduced minimization problems :

Problem 1

Minimizing (5) by fixing \(\mathrm {\textbf{R}}\) to compute \(\mathrm {\textbf{W}}\) (for feature selection). To solve this problem, we consider the lagrangian function of (5):

$$\begin{aligned} \mathcal {L}_\mathrm {\textbf{W}}=trace(\mathrm {\textbf{W}}^T{{\,\mathrm{\textbf{E}}\,}}^T{{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}-2\mathrm {\textbf{W}}^T{\textbf {E}}^T({\textbf {R}}^T+{\textbf {Z}}))+\lambda \parallel \mathrm {\textbf{W}}\parallel _{2,1}. \end{aligned}$$
(7)

Then, we calculate the derivative of \(\mathcal {L}_W\) w.r.t \(\mathrm {\textbf{W}}\):

$$\begin{aligned} \frac{\partial \mathcal {L}_\mathrm {\textbf{W}}}{\partial \mathrm {\textbf{W}}}=2{{\,\mathrm{\textbf{E}}\,}}^T{{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}-2{{\,\mathrm{\textbf{E}}\,}}^T(\mathrm {\textbf{R}}^T+{\textbf {Z}})+2\lambda \mathcal {D}_\mathrm {\textbf{W}}\mathrm {\textbf{W}}. \end{aligned}$$
(8)

Where \(\mathcal {D}_W\) is a (\(m\times m\)) diagonal matrix with the \(i^{th}\) element equal to \(\frac{1}{2\parallel {\textbf {W}}(i,:)\parallel _2 }\). Subsequently, we set the derivative to zero to update \(\mathrm {\textbf{W}}\):

$$\begin{aligned} \mathrm {\textbf{W}}= ({\textbf {E}}^T{\textbf {E}}+\lambda \mathcal {D}_\mathrm {\textbf{W}})^{-1}{{\,\mathrm{\textbf{E}}\,}}^T(\mathrm {\textbf{R}}^T+{\textbf {Z}}) \end{aligned}$$
(9)

Problem 2

Minimizing (5) by fixing \(\mathrm {\textbf{W}}\) to compute the solution for \(\mathrm {\textbf{R}}\) (for explanation selection). To solve this problem, we consider the Lagrangian function of (5):

$$\begin{aligned} \mathcal {L}_\mathrm {\textbf{R}}=trace(\mathrm {\textbf{R}}^T\mathrm {\textbf{R}}-2\mathrm {\textbf{R}}^T({{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}-{\textbf {Z}}))+\beta \parallel \mathrm {\textbf{R}}\parallel _{2,1}. \end{aligned}$$
(10)

Then, we calculate the derivative of \(\mathcal {L}_\mathrm {\textbf{R}}\) w.r.t \(\mathrm {\textbf{R}}\):

$$\begin{aligned} \frac{\partial \mathcal {L}_\mathrm {\textbf{R}}}{\partial \mathrm {\textbf{R}}}=2\mathrm {\textbf{R}}^T-2({{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}-{\textbf {Z}})+2\beta \mathcal {D}_\mathrm {\textbf{R}}\mathrm {\textbf{R}}^T. \end{aligned}$$
(11)

Where \(\mathcal {D}_\mathrm {\textbf{R}}\) is a (\(n\times n\)) diagonal matrix with the \(i^{th}\) element equal to \(\frac{1}{2\parallel \mathrm {\textbf{R}}^T(i,:)\parallel _2 }\).

Subsequently, we set the derivative to zero to update B:

$$\begin{aligned} \mathrm {\textbf{R}}= ({{\,\mathrm{\textbf{E}}\,}}\mathrm {\textbf{W}}-{\textbf {Z}})^T(({\textbf {I}} + \beta \mathcal {D}_\mathrm {\textbf{R}})^{-1})^T \end{aligned}$$
(12)

Where I is a (\(n\times n\)) identity matrix. All of the above developments are summarized on Algorithm 3.

Algorithm 3
figure c

Co-Selection pick (CoSP).

Full size image
Fig. 2
figure 2

Explanation matrix \({{\,\mathrm{\textbf{E}}\,}}\) (this matrix is given as input for CoSP Algorithm 3)

Full size image

4.5 Algorithm analysis

In the Algorithm 3, the final user expects a selection of \(\mathrm {\textbf{B}}\) instances (e.g., explanations) and \(\mathrm {\textbf{L}}\) features which are most relevant to provide global explanation of the model. In order to achieve this, CoSP requires four hyper-parameters \(\lambda \), \(\beta \) , \(\delta \) and h that will be used later on to build the set of chosen instances and features. Firstly, we build the explanations matrix \({{\,\mathrm{\textbf{E}}\,}}\) using any explainer, in our case we use LIME. Secondly, we compute the similarity matrix \(\mathrm {\textbf{A}}\) either in supervised mode (as adjacency matrix or a binary matrix) or in an unsupervised way according to the availability of the labels of the test instances \(\mathrm {\textbf{X}}\). Then, we eigen-decompse \(\mathrm {\textbf{A}}\) to find Z. From line 9 to line 13 \(\mathrm {\textbf{W}}\) and \(\mathrm {\textbf{R}}\) are updated until convergence according to (9) and (12). Following the alternate optimization, we rank the instances and the features according to \(\mathrm {\textbf{R}}\) and \(\mathrm {\textbf{W}}\) respectively. So, the higher the norm of \(\parallel {\textbf {R}}(:,j)\parallel _2\), the more the \(j^{th}\) explanation is not representative, while the higher the norm \(\parallel {\textbf {W}}(i,:)\parallel _2\), the more the \(i^{th}\) feature is important. The computational complexity of Algorithm 3 is presented by the following lemma.

Lemma

CoSP is computed in time of \(\mathcal {O}\) \((nmh+m^3+n^3+nm^2+n^2h)\).

Proof

The time complexity of CoSP essentially depends on the rule of (9) as well as the rule of (12). These two rules are for updating the two matrices W and R which consists of some matrix multiplication and inversion operations at each iteration. Specifically, the computation of the derivative w.r.t W requires \(\mathcal {O}\) \((nmh+nm^2+m^3)\). The derivative w.r.t R needs \(\mathcal {O}\) \((nmh+n^2h+n^3)\).

5 Experiments

In this section, we conduct some experiments to validate our framework on some known sentiment datasets.

5.1 Datasets and compared methods

We use a binary sentimental classification dataset. Sentimental analysis is the task of analyzing people’s opinions, reviews, and comments presented as textual data. It gives intuition about different points of view and feedback by detecting relevant words used to express specific sentiments [53]. Today, companies rely on sentimental analysis to improve their strategy. People’s opinions are collected from different sources like Facebook, Tweets, product reviews and processed in order to understand customer’s needs and improve marketing plans. When the sentiment is divided into positive and negative ones, it is called binary sentimental analysis which is the most common type and the one used in our case. While multi-class sentiment analysis classifies text into groups of possible labels. We use multi-Domain Sentiment DatasetFootnote 1, which contains multiple domains reviews (books and dvd) from Amazon.com, where for each type of product there are hundred of thousands of collected reviews. Then, we use an experiment introduced in [11] which aims to evaluate if explanations could help a simulated user to recognize the best model from a group of models having the same accuracy on validation set. In order to do this, a new dataset will be generated by adding 10 artificial features to the train and validation set from original public dataset (reviews). For the train examples, each of those features appears in 10% of instances in one class and in 20% of the other class. In the test examples, an artificial feature appears in 10 % of examples in both classes. This represents the case of having spurious correlations in the data introduced by non informative features.

Furthermore, we train pairs of classifiers until their validation accuracy is within 0.1% of each other. However, their test accuracy should differ by at least 5% which will make one classifier better than the other. Then, we explain global behaviors of both classifiers using our proposed approach CoSP.

We compare CoSP against Random, Greedy [25], Parzen [26] and LIME approches [11] combined with Submodular Pick (SP) and Random Pick (RP). In the following, we briefly describe each approach.

  • Random randomly chooses the features as an explanation.

  • Greedy removes features highly contributing to the predicted class until the prediction changes.

  • Parzen uses parzen windows to globally approximate the classifier

  • Lime explains the classifier predictions by approximating it locally with an interpretable model.

In the experiment, the explanations were obtained with the above four local explainability techniques. Then, the global explainability approaches CoSP, SP or RP were used to select the relevant instances.

5.2 Experimental setting

To validate our approach, we use the same experimental setting introduced in [54] by selecting top five important features per class chosen as most relevant ones to be considered for the classification task. Global approach is validated if it selects distinguishing features. Four hyper-parameters necessary for CoSP have been set as follows: \(\lambda \approx 2.11\), \(\beta \approx 61.79\) , \(\delta = 1\) and \(h = 17000\) (which stands for the number of features selected by CoSP). Parallely, the parameters configuration of compared methods is as follows: K = 10 words in each explanation and B = 10 instances.

5.3 Evaluation and results

In this section, we present the main results of our experiments. Figures 3, 4, 5 and 6 show the experimental results over two datasets, Books and DVDs. We summarize the main observations of the experimental results in the following points.

Fig. 3
figure 3

Accuracy of picking the correct classifier over two datasets Books and DVDs

Full size image
Fig. 4
figure 4

Top 5 features per class picked by CoSP global approach for review’s binary classification on books dataset

Full size image
Fig. 5
figure 5

Top 5 features per class picked by CoSP global approach for review’s binary classification on DVDs dataset

Full size image
Fig. 6
figure 6

Accuracy of human subject in choosing between two models, using books dataset. The four method are combined with CoSP or Submodular (SP) selections

Full size image

  • In terms of Accuracy, the LIME combined with either Co-Selection Pick (CoSP) or Submodular Pick (SP-LIME) outperforms other comparison algorithms. It means that the explanations provided by LIME are faithful to the models (see Figure 6).

  • Regardless of the choice of the explainers, CoSP is significantly better than SP or RP, across the two data sets, followed by SP-LIME (see Figures 3 and 6).

  • CoSP further improves the user’s ability to select the best classifier comparing with the SP or RP (see Figures 3 and 6).

  • From Figures 4 and 5, the displayed perception contains words that are meaningful in order to judge the type of comment. Features are aligned with human intuition and words with no representative meaning like stop words were not selected. Also, noisy features labeled with prefix “FAKE” added to the dataset were not deemed important.

6 Conclusion

In this paper, we presented CoSP, a generic framework aiming to select individual instances in order to provide global explanation for machine learning models. We used Co-selection based on similarity as foundation to build global understanding of the black box internal logic over any local explainer. Furthermore, we conducted some experiments showing that CoSP offers representative insights. This study is a another step towards understanding machine learning models globally. For future work, we would like to explore this method in the context of time series data, as it is a challenging to find representative illustration for this type of data. The approach we proposed is independent of the type of data, since it is based on the explanations provided by a local explainer. Concerning time series, the local explainer must be capable of processing this type of data. This involves in particular the choice of an efficient representation of the time series. In the case of LIME, it is necessary to find a vector representation of the series to be able to apply LASSO and have the explanations. Among the applications on which we want to apply our approach, there is the detection of contextual anomalies in time series. The idea is then not only to detect abnormal segments in a time series but to explain why such a segment was detected as abnormal.