Keywords

1 Introduction

It is well-known that many real-life data sets are incomplete, i.e., are affected by missing attribute values. Recently many papers presenting a rough set approach in research on incomplete data were published, see, e.g., [4, 7, 9–17, 21–27, 31–34, 37, 38, 40–42, 44, 47–62, 68–72, 74–78, 80, 81].

Most of the rough set activity in research on incomplete data is conducted in data mining. Using a rough set approach to incomplete data, we may distinguish between different interpretations of missing attribute values.

If an attribute value was accidentally erased or is unreadable, we may use the most cautious approach to missing attribute values and mine data using only specified attribute values. The corresponding type of missing attribute values is called lost and is denoted by “?”. Mining incomplete data affected by lost values was studied for the first time in [44], where two algorithms for rule induction from such data were presented. The same data sets were studied later, see, e.g., [76, 77].

Another type of missing attribute values happens when a respondent refuses to answer a question that seems to be irrelevant. For example, a patient is tested for a disease and one of the questions is a color of hair. The respondent may consider the color of hair to be irrelevant. This type of missing attribute values is called a “do not care” condition and is denoted by “*”. The first study of “do not care” conditions, again using rough set theory, was presented in [17], where a method for rule induction in which missing attribute values were replaced by all values from the domain of the attribute was introduced. “Do not care” conditions were also studied later, see, e.g. [50, 51].

In yet another interpretation of missing attribute values, called an attribute-concept value, and denoted by “\(-\)”, we assume that we know that the corresponding case belongs to a specific concept X and, as a result, we may replace the missing attribute value by attribute values for all cases from the same concept X. A concept (class) is a set of all cases classified (or diagnosed) the same way. For example, if for a patient the value of an attribute Temperature is missing, this patient is sick with Flu, and all remaining patients sick with Flu have Temperature values high then using the interpretation of the missing attribute value as the attribute-concept value, we will replace the missing attribute value by the value high. This approach was introduced in [24].

An approach to mining incomplete data presented in this paper is based on the idea of an attribute-value block. A characteristic set, defined as an intersection of such blocks, is a generalization of the elementary set, well-known in rough set theory [63–65]. A characteristic relation, defined by characteristic sets, is, in turn, a generalization of the indiscernibility relation. As it was shown in [21], incomplete data are described by three different types of approximations: singleton, subset and concept.

For rule induction from incomplete data it is the most natural to use the MLEM2 (Modified Learning form Examples Module, version 2) [2, 18–20] since this algorithm is also based on attribute-value pair blocks. A number of extensions of this algorithm were developed in order to process incomplete data sets using different definitions of approximations, see, e.g., [5, 31, 43, 45].

One of the fundamental concepts of rough set theory is lower and upper approximations. A generalization of such approximations, a probabilistic approximation, introduced in [79], was applied in variable precision rough set models, Bayesian rough sets and decision-theoretic rough set models [46, 66, 67, 73, 82–86]. These probabilistic approximations are defined using the indiscernibility relation. For incomplete data, probabilistic approximations were extended to characteristic relation in [30]. The probabilistic approximation is associated with some parameter \(\alpha \) (interpreted as a probability). If \(\alpha \) is very small, say 1 / |U|, where U is the set of all cases, the probabilistic approximation is reduced to the upper approximation; if \(\alpha \) is equal to 1.0, the probabilistic approximation is equal to the lower approximation. Local probabilistic approximations, based on attribute-value blocks instead of characteristic sets, were defined in [7], see also [31].

2 Fundamental Concepts

A basic tool to analyze incomplete data sets is a block of an attribute-value pair. Let (a, v) be an attribute-value pair. For complete data sets, i.e., data sets in which every attribute value is specified, a block of (a, v), denoted by [(a, v)], is the set of all cases x for which \(a(x) = v,\) where a(x) denotes the value of the attribute a for the case x. For incomplete data sets the definition of a block of an attribute-value pair is modified.

  • If for an attribute a there exists a case x such that \(a(x) = \ ?\), i.e., the corresponding value is lost, then the case x should not be included in any blocks [(a, v)] for all values v of attribute a,

  • If for an attribute a there exists a case x such that the corresponding value is a “do not care” condition, i.e., \(a(x) = *\), then the case x should be included in blocks [(a, v)] for all specified values v of attribute a.

  • If for an attribute a there exists a case x such that the corresponding value is an attribute-concept value, i.e., \(a(x) = -\), then the corresponding case x should be included in blocks [(a, v)] for all specified values \(v \in V(x, a)\) of attribute a, where

    $$\begin{aligned} V(x, a) = \{a(y) \ | \ a(y) \ is \ specified,&y \in U, \ d(y) = d(x)\}. \end{aligned}$$

For a case \(x \in U\) the characteristic set \(K_B(x)\) is defined as the intersection of the sets K(x, a), for all \(a \in B\), where the set K(x, a) is defined in the following way:

  • If a(x) is specified, then K(x, a) is the block [(a, a(x)] of attribute a and its value a(x),

  • If \(a(x)) = ?\) or \(a(x) = *\), then the set \(K(x, a) = U\).

  • If \(a(x) = -\), then the corresponding case x should be included in blocks [(a, v)] for all known values \(v \in V(x, a)\) of attribute a. If V(x, a) is empty, \(K(x, a) = U.\)

The characteristic relation R(B) is a relation on U defined for \(x, y \in U\) as follows

$$\begin{aligned} (x, y) \in R(B) \ if \ and \ only \ if \ y \in K_B(x). \end{aligned}$$

The characteristic relation R(B) is reflexive but—in general—does not need to be symmetric or transitive.

3 Lower and Upper Approximations

For incomplete data sets there is a few possible ways to define approximations [24, 43]. Let X be a concept, let B be a subset of the set A of all attributes, and let R(B) be the characteristic relation of the incomplete decision table with characteristic sets \(K_B(x)\), where \(x \in U\). A singleton B-lower approximation of X is defined as follows:

$$\begin{aligned} \underline{B}X = \{x \in U \ | \ K_{B}(x) \subseteq X \}. \end{aligned}$$

A singleton B-upper approximation of X is

$$\begin{aligned} \overline{B}X = \{x \in U \ | \ K_B(x) \cap X \ne \emptyset \}. \end{aligned}$$

We may define lower and upper approximations for incomplete data sets as unions of characteristic sets. There are two possibilities. Using the first way, a subset B-lower approximation of X is defined as follows:

$$\begin{aligned} \underline{B}X = \cup \{K_B(x) \ | \ x \in U, K_B(x) \subseteq X \}. \end{aligned}$$

A subset B-upper approximation of X is

$$\begin{aligned} \overline{B}X = \cup \{K_B(x) \ | \ x \in U, K_B(x)\cap X \ne \emptyset \}. \end{aligned}$$

The second possibility is to modify the subset definition of lower and upper approximation by replacing the universe U from the subset definition by a concept X. A concept B-lower approximation of the concept X is defined as follows:

$$\begin{aligned} \underline{B}X = \cup \{K_B(x) \ | \ x \in X, K_B(x) \subseteq X \}. \end{aligned}$$

The subset B-lower approximation of X is the same set as the concept B-lower approximation of X. A concept B-upper approximation of the concept X is defined as follows:

$$\begin{aligned}&\overline{B}X = \cup \{K_B(x) \ | \ x \in X, K_B(x)\cap X \ne \emptyset \} = \cup \{K_B(x) \ | \ x \in X \}. \end{aligned}$$

Two traditional methods of handling missing attribute values: Most Common Value for symbolic attributes and Average Values for numeric attributes (MCV-AV) and Concept Most Common Values for symbolic attributes and Concept Average Values for numeric attributes (CMCV-CAV), for details see [36], were compared with three rough-set interpretations of missing attribute values: lost values, attribute-concept values and “do not care” conditions combined with concept lower and upper approximations in [27]. In turned out that there is no significant difference in performance in terms of an error rate measured by ten-fold cross validation between the traditional and rough-set approaches to missing attribute values.

In [26] the same two traditional methods, MCV-AV and CMCV-CAV, and other two traditional methods (Closest Fit and Concept Closest Fit), for details see [36], were compared with the same three rough set interpretations of missing attribute values combined with concept approximations. The best methodology was based on the Concept Closest Fit combined with rough set interpretation of missing attribute values as lost values and concept lower and upper approximations.

Additionally, in [28], a CART approach to missing attribute values [1] was compared with missing attribute values interpreted as lost values combined with concept lower and upper approximations. In two cases CART was better, in two cases rough set approach was better, and in one case the difference was insignificant. Hence both approaches are comparable in terms of an error rate.

In [29, 39], the method CMCV-CAV was compared with rough set approaches to missing attribute values, the conclusion was that CMCV-CAV was either worse or not better, depending on a data set, than rough-set approaches.

4 Probabilistic Approximations

In this section we will extend definitions of singleton, subset and concept approximations to corresponding probabilistic approximations. The problem is how useful are proper probabilistic approximations (with \(\alpha \) larger than 1 / |U| but smaller than 1.0). We studied usefulness of proper probabilistic approximations for incomplete data sets [3], where we concluded that proper probabilistic approximations are not frequently better than ordinary lower and upper approximations.

A B-singleton probabilistic approximation of X with the threshold \(\alpha \), \(0 < \alpha \le 1\), denoted by \(appr_{\alpha ,B}^{singleton}(X)\), is defined by

$$\begin{aligned} \{x \ | \ x \in U, \ Pr(X \ | \ K_B(x)) \ge \alpha \}, \end{aligned}$$

where \(Pr(X \ | \ K_B(x)) = \frac{|X \ \cap \ K_B(x)|}{| K_B(x)|}\) is the conditional probability of X given \( K_B(x)\) and |Y| denotes the cardinality of set Y.

A B-subset probabilistic approximation of the set X with the threshold \(\alpha \), \(0 < \alpha \le 1\), denoted by \(appr_{\alpha , B}^{subset} (X)\), is defined by

$$\begin{aligned} \cup \{K_B(x) \ | \ x \in U, \ Pr(X \ | \ K_B(x)) \ge \alpha \} \end{aligned}$$

A B-concept probabilistic approximation of the set X with the threshold \(\alpha \), \(0 < \alpha \le 1\), denoted by \(appr_{\alpha , B}^{concept} (X)\), is defined by

$$\begin{aligned} \cup \{K_B(x) \ | \ x \in X, \ Pr(X \ | \ K_B(x)) \ge \alpha \}. \end{aligned}$$

In [6] ordinary lower and upper approximations (singleton, subset and concept), special cases of singleton, subset and concept probabilistic approximations, were compared with proper probabilistic approximations (singleton, subset and concept) on six data sets with missing attribute values interpreted as lost values and “do not care” conditions, in terms of an error rate. Since we used six data sets, two interpretations of missing attribute values and three types of probabilistic approximations, there were 36 combinations. Among these 36 combinations, for five combinations the error rate was smaller for proper probabilistic approximations than for ordinary (lower and upper) approximations, for other four combinations, the error rate for proper probabilistic approximations was larger than for ordinary approximations, for the remaining 27 combinations, the difference between these two types of approximations was not statistically significant.

Results of experiments presented in [9] show that among all probabilistic approximations (singleton, subset and concept) and two interpretations of missing attribute values (lost values and “do not care” conditions) there is not much difference in terms of an error rate measured by ten-fold cross validation. On the other hand, complexity of induced rule sets differs significantly. The simplest rule sets (in terms of the number of rules and the total number of conditions in the rule set) we accomplished by using subset probabilistic approximations combined with “do not care” conditions.

In [8] results of experiments using all three probabilistic approximations (singleton, subset and concept) and two interpretations of missing attribute values: lost values and “do not care” conditions were compared with the MCV-AV and CMCV-CAV methods in terms of an error rate. For every data set, the best among six rough-set methods (combining three kinds of probabilistic approximations with two types of interpretations of missing attribute vales) were compared with the better results of MCV-AV and CMCV-CAV. Rough-set methods were better for five (out of six) data sets.

5 Local Approximations

An idea of the local approximation was introduced in [41]. A local probabilistic approximation was defined in [7]. A set T of attribute-value pairs, where all attributes belong to the set B and are distinct, is called a B-complex. In the most general definition of a local probabilistic definition we assume only an existence of a family \(\mathcal {T}\) of B-complexes T with the conditional probability Pr(X|[T]) of \(X \ge \alpha \), where \(Pr(X|[T]) = \frac{|X \cap [T]|}{| [T]|}\).

A B-local probabilistic approximation of the set X with the parameter \(\alpha \), \(0 < \alpha \le 1\), denoted by \(appr_{\alpha }^{local} (X)\), is defined as follows

$$\begin{aligned} \cup \{[T] \ | \ \exists \ a \ family \ \mathcal {T} \ of \ B-complexes \ T \ of \ X \ with \ \forall \ T \in \mathcal {T}, \ Pr(X|[T]) \ge \alpha \} . \end{aligned}$$

In general, for given set X and \(\alpha \), there exists more than one A-local probabilistic approximation. However, for given set X and parameter \(\alpha \), a B-local probabilistic approximation given by the next definition is unique.

A complete B-local probabilistic approximation of the set X with the parameter \(\alpha \), \(0 < \alpha \le 1\), denoted by \(appr_{\alpha }^{complete} (X)\), is defined as follows

$$\begin{aligned} {\cup \{[T] \ | \ T \ is \ a \ B-complex \ of \ X, \ Pr(X|[T]) \ge \alpha \} }. \end{aligned}$$

Due to computational complexity of determining complete local probabilistic approximations, yet another local probabilistic approximation, called a MLEM2 local probabilistic approximation and denoted by \(appr_{\alpha }^{mlem2} (X)\), is defined by using A-complexes Y that are the most relevant to X, i.e., with \(|X \cap Y|\) as large as possible, etc., following the MLEM2 algorithm.

In [31] concept probabilistic approximations were compared with complete local probabilistic approximations and with MLEM2 local probabilistic approximations for eight data sets, using two interpretations of missing attribute values: lost vales and “do not care” conditions, in terms of an error rate. Since two interpretations of missing attribute values and eight data sets were used, there were 16 combinations. There was not clear winner among three kinds of probabilistic approximations. In four combinations the best was the concept probabilistic approximation, in three cases the best was the complete local probabilistic approximation, and in four cases the best was the MLEM2 local probabilistic approximation. For remaining five combinations the difference in performance between all three approximations was insignificant.

6 Special Topics

When replacing existing, specified attribute values by symbols of missing attribute values, e.g., by “?”s, an error rate computed by ten-fold cross validation may be smaller than for the original, complete data set. Thus, increasing incompleteness of the data set may improve accuracy. Results of experiments showing this phenomenon were published, e.g., in [34, 35].

Yet another problem is associated with consistency. For complete data sets, a data set is consistent if any two cases, indistinguishable by all attributes, belong to the same concept. The idea of consistency is more complicated for incomplete data. This problem was discussed in [4] and also in [54, 60, 72].

7 Conclusions

Research on incomplete data is very active and promising, with many open problems and potential for additional progress.