Abstract
Learning from non-stationary data presents several new challenges. Among them, a significant problem comes from the sudden changes in the incoming data distributions, the so-called concept drift. Several concept drift detection methods exist, generally based on distances between distributions, either arbitrarily selected or context-dependent. This paper presents a straightforward approach for detecting concept drift based on a weighted dissimilarity metric over posterior probabilities. We also evaluate the performance of three well-known dissimilarity metrics when used by the proposed approach. Experimental evaluation has been done over ten datasets with injected sudden drifts in a binary classification context. Our results first suggest choosing the Kullback-Leibler divergence, and second, they show that our drift detection procedure based on dissimilarity measures is pretty efficient.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sobolewski, P., Woźniak, M.: Concept drift detection and model selection with simulated recurrence and ensembles of statistical detectors. J. Univers. Comput. Sci. 19(4), 462–483 (2013)
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv., 46(4), 441–4437 (2014)
Hinder, F., Vaquet, V., Hammer, B.: Suitability of Different Metric Choices for Concept Drift Detection, Arxiv (2022)
Basterrech, S., Woźniak, M.: Tracking changes using Kullback-Leibler divergence for the continual learning. IEEE SMC’2022. ArXiv (2022)
Ditzler, G., Polikar, R.: Hellinger distance based drift detection for nonstationary environments. In IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE), 41–48, (2011)
Brzezinski, D., Stefanowski, J.: Reacting to different types of concept drift: the accuracy updated ensemble algorithm. IEEE Trans. Neural Netw. Lear. Syst. 25(1), 81–94 (2014)
Gonzalvez, P.M., Santos, S.D.C., Barros, R., Vieira, D.: A comparative study on concept drift detectors. Expert Syst. Appl. 41(18), 8144–8156 (2014)
Gustafsson, F.: Adaptive Filtering and Change Detection. Wiley, (2000)
Dasu, T., Krishnan, S., VenkataSubramanian, S.: An information-theoretic approach to detecting changes in multidimensional data streams. Interfaces, 1–24, (2006)
Faber, K., Corizzo, R., Sniezynski, B., Baron, M., Japkowicz, N.: WATCH: wasserstein Change Point Detection for High-Dimensional Time Series Data. In: 2021 IEEE Int. Conf. on Big Data (Big Data), 4450–4459 (2021)
Igor, G., Webb, G.: Survey of distance measures for quantifying concept drift and shift in numeric data. Know. Inf. Syst. 591–615, (2019)
Cover, T.M., Thomas, J.A.: Elements of information theory. John Wiley Sons, (2012)
Gibbs, A.L., Su, F, E: On choosing and bounding probability metrics. Available in ArXiv, (2002)
Basterrech, S., Krömer, P.: A nature-inspired biomarker for mental concentration using a single-channel EEG. Neural Comput. Appl. (2019)
Basterrech, S., Bobrov, P., Frolov, A., Húsek, D.: Nature-Inspired Algorithms for Selecting EEG Sources for Motor Imagery Based BCI. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2015. LNCS (LNAI), vol. 9120, pp. 79–90. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19369-4_8
Ksieniewicz, P., Zyblewski, P.: stream-learn—open-source python library for difficult data stream batch analysis. Neurocomputing (2022)
Acknowledgements
This work was supported by the CEUS-UNISONO programme, which has received funding from the National Science Centre, Poland under grant agreement No. 2020/02/Y/ST6/00037, and the GACR-Czech Science Foundation project No. 21-33574K “Lifelong Machine Learning on Data Streams”.
It was also supported by the ClimateDL project (code 22-CLIMAT-02) belonging to the Climate AmSud programme, where the central problem is forecasting extreme temperatures in future periods such as in the following summer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Basterrech, S., Platoš, J., Rubino, G., Woźniak, M. (2023). Experimental Analysis on Dissimilarity Metrics and Sudden Concept Drift Detection. In: Abraham, A., Pllana, S., Casalino, G., Ma, K., Bajaj, A. (eds) Intelligent Systems Design and Applications. ISDA 2022. Lecture Notes in Networks and Systems, vol 716. Springer, Cham. https://doi.org/10.1007/978-3-031-35501-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-35501-1_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35500-4
Online ISBN: 978-3-031-35501-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)