Rare Pattern Mining from Data Stream Using Hash-Based Search and Vertical Mining

Abstract

Rare itemset mining is the emerging research domain in data mining. Patterns with low support and high confidence are referred to as Rare Patterns, which are very interesting compared to frequent Patterns in certain application domains like analysis of network logs, online customer purchase behavior, online banking transaction analysis, sensor data analysis, stock market data analysis. Many applications generate large volumes of the continuous data streams. To analyze such data streams and to identify rare patterns, we need efficient algorithms that can process data streams. Many research articles on rare pattern mining are available for static databases. However, it is not possible to apply the algorithms designed for static databases to data streams. Hence, we need algorithms that are specifically designed for data stream processing, to mine important rare patterns. Rare pattern mining from the data stream is in the budding stage and only a few algorithms are available. To address this, we have proposed algorithm HEclat-RPStream, an Eclat based method to mine rare patterns from a data stream using a vertical mining with bitsets. The discovered patterns are maintained in a prefix-based rare pattern tree, it uses double hashing to maintain a rare pattern in the data stream. The algorithm also uses the Breadth First Search (BFS) and Depth First Search (DFS) to discover interesting large itemsets. To handle data streams, we have used a time sensitive sliding window approach which captures most recent patterns. The pruning technique based on two items is used to optimize the performance. The experimental results of the proposed method demonstrated good performance concerning execution time and the total number of rare patterns generated.