Abstract
Forecasting the particulate matter (PM) concentration in South Korea has become urgently necessary owing to its strong negative impact on human life. In most statistical or machine learning methods, independent and identically distributed data, for example, a Gaussian distribution, are assumed; however, time series such as air pollution and weather data do not meet this assumption. In this study, the detrended fluctuation analysis and power-law analysis are used in an analysis of the statistical characteristics of air pollution and weather data. Rigorous seasonality adjustment of the air pollution and weather data was performed because of their complex seasonality patterns and the heavy-tailed distribution of data even after deseasonalization. The maximum correntropy criterion for regression (MCCR) loss was applied to multiple models including conventional statistical models and state-of-the-art machine learning models. The results show that the MCCR loss is more appropriate than the conventional mean squared error loss for forecasting extreme values.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Abbreviations
- C(r):
-
Autocorrelation
- F(x):
-
Cumulative distribution function
- \(\overline F (x)\) :
-
Complementary cumulative distribution function
- h :
-
DFA fluctuation exponent
- M i :
-
Predicted value
- O i :
-
Actual value
- res h :
-
Residuals
- s y, smoothed :
-
Smoothed yearly seasonality
- s w :
-
Weekly seasonality
- s h :
-
Daily seasonality
- V(s):
-
DFA fluctuation function
- x(t):
-
Single column input time series
- Y t :
-
Actual value
- \({\overline Y _t}\) :
-
Predicted value
- α :
-
Pareto index
- β :
-
MCCR scale parameter
- ξ :
-
Long-range dependence power law exponent
References
EPA, Health and Environmental Effects of Particulate Matter (PM), United States Environmental Protection Agency, United States Government (2016).
H. B. Kim et al., Long-term exposure to air pollutants and cancer mortality: a meta-analysis of cohort studies, Int. J. Environ. Res. Public Health, 15(11) (2018) 2608.
C. A. Pope and D. W. Dockery, Health effects of fine particulate air pollution: lines that connect, J. Air Waste Manag. Assoc., 56(6) (2006) 709–742.
OECD, The Economic Consequences of Outdoor Air Pollution, OECD Publishing, Paris (2016).
AirKorea, Annual Report of Air Quality, Korea Environment Corporation (2019).
WHO, Air Quality Guidelines, World Health Organization (2006).
OECD, Air Pollution Exposure (Indicator), Organization for Economic Co-operation and Development (2021).
P. A. Makar et al., Feedbacks between air pollution and weather, part 1: effects on weather, Atmos. Environ., 115 (2015) 442–469.
I. Bouarar et al., Influence of anthropogenic emission inventories on simulations of air quality in China during winter and summer 2010, Atmos. Environ., 198 (2019) 236–256.
H. J. Lee et al., Impacts of atmospheric vertical structures on transboundary aerosol transport from China to South Korea, Sci. Rep., 9 (2019) 13040.
C. Jordan et al., Investigation of factors controlling PM25 variability across the South Korean Peninsula during KORUSAQ, Elem. Sci. Anth., 8 (28) (2020).
K. P. Singh et al., Linear and nonlinear modeling approaches for urban air quality prediction, Sci. Total Environ., 426 (2012) 244–255.
Y. Zhang et al., Real-time air quality forecasting, part I: history, techniques, and current status, Atmos. Environ., 60 (2012) 632–655.
D. W. Byun and J. K. S. Ching, Science Algorithms of the EPA Models-3 Community Multiscale Air Quality (CMAQ) Modeling System, EPA/600/R-99/030, United States Environmental Protection Agency, United States Government (1999).
I. Bey et al., Global modeling of tropospheric chemistry with assimilated meteorology: model description and evaluation, J. Geophys. Res. Atmos, 106(D19) (2001) 23073–23095.
G. A. Grell et al., Fully coupled online chemistry within the WRF model, Atmos. Environ., 39(37) (2005) 6957–6975.
A. Baklanov et al., Towards improving the simulation of meteorological fields in urban areas through updated/advanced surface fluxes description, Atmos. Chem. Phys., 8(3) (2008) 523–543.
G. Zhou et al., Numerical air quality forecasting over eastern China: an operational application of WRF-Chem, Atmos. Environ., 153 (2017) 94–108.
Z. Shang et al., A novel model for hourly PM25 concentration prediction based on CART and EELM, Sci. Total Environ., 651 (2019) 3043–3052.
D. Guo, R. Guo and C. Thiart, Predicting air pollution using fuzzy membership grade Kriging, Comput. Environ. Urban Syst., 31(1) (2007) 33–51.
W. Wang and Y. Guo, Air pollution PM25 data analysis in Los Angeles long beach with seasonal ARIMA model, 2009 Int. Conf. Energy Environ. Technol. (2009) 7–10.
G. Lai et al., Modeling long- and short-term temporal patterns with deep neural networks, 41st Int. ACM SIGIR Conf. Res. Dev. Inf. Retrieval (2018).
S. Y. Shih, F. K. Sun and H. Y. Lee, Temporal pattern attention for multivariate time series forecasting, Mach. Learn., 108(8) (2019) 1421–1441.
S. Li et al., Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, arXiv: 1907.00235 (2019).
J. E. Choi, H. Lee and J. Song, Forecasting daily PM10 concentrations in Seoul using various data mining techniques, Commun. Stat. Appl. Methods, 25(2) (2018) 199–215.
K. Cho et al., Air quality prediction using a deep neural network model, J. Korean Soc. Atmos. Environ., 35(2) (2019) 214–225.
F. Franceschi, M. Cobo and M. Figueredo, Discovering relationships and forecasting PM10 and PM25 concentrations in Bogotá, Colombia, using artificial neural networks, principal component analysis, and k-means clustering, Atmos. Pollut. Res., 9(5) (2018) 912–922.
Y. Bai et al., Hourly PM25 concentration forecast using stacked autoencoder model with emphasis on seasonality, J. Clean. Prod., 224 (2019) 739–750.
W. Liu, P. P. Pokharel and J. C. Principe, Correntropy: properties and applications in non-Gaussian signal processing, IEEE Trans. Signal Process., 55(11) (2007) 5286–5298.
Y. Feng et al., Learning with the maximum correntropy criterion induced losses for regression, J. Mach. Learn. Res., 16(30) (2015) 993–1034.
R. Cichowicz, G. Wielgosiński and W. Fetter, Dispersion of atmospheric air pollution in summer and winter season, Environ. Monit. Assess., 189(12) (2017) 605.
O. Troyanskaya et al., Missing value estimation methods for DNA microarrays, Bioinformatics, 17(6) (2001) 520–525.
F. Pedregosa et al., Scikit-learn: machine learning in python, J. Mach. Learn. Res., 12 (2011) 2825–2830.
W. S. Cleveland, Robust locally weighted regression and smoothing scatterplots, J. Am. Stat. Assoc., 74(368) (1979) 829–836.
J. W. Kantelhardt et al., Detecting long-range correlations with detrended fluctuation analysis, Phys. A Stat. Mech. Its Appl., 295(3–4) (2001) 441–454.
E. Koscielny-Bunde et al., Long-term persistence and multifractality of river runoff records: detrended fluctuation studies, J. Hydrol., 322(1–4) (2006) 120–137.
A. Clauset, C. R. Shalizi and M. E. J. Newman, Power-law distributions in empirical data, SIAM Rev., 51(4) (2009) 661–703.
J. Alstott, E. Bullmore and D. Plenz, Powerlaw: a python package for analysis of heavy-tailed distributions, PLoS One, 9(1) (2014) e85777.
G. E. Uhlenbeck and L. S. Ornstein, On the theory of the Brownian motion, Phys. Rev., 36 (1930) 823.
M. T. Wojnowicz, The Ornstein-Uhlenbeck process in neural decision-making: mathematical foundations and simulations suggesting the adaptiveness of robustly integrating stochastic neural evidence, Master’s Thesis, University of Washington (2012).
R. J. Hyndman and G. Athanasopoulos, Forecasting: Principles and Practice, Second Edition, OTexts, Melbourne, Australia (2018).
T. Chen and C. Guestrin, XGBoost: a scalable tree boosting system, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2016) 785–794.
I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press (2015).
G. Zhang, B. Eddy Patuwo and M. Y. Hu, Forecasting with artificial neural networks: the state of the art, Int. J. Forecast., 14(1) (1998) 35–62.
S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput., 9(8) (1997) 1735–1780.
K. Cho et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation, Proc. 2014 Conf. Empir. Methods Nat. Lang. Process. (2014) 1724–1734.
D. Bahdanau, K. H. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, 3rd Int. Conf. Learn. Represent. ICLR 2015 — Conf. Track Proc. (2015).
N. Wu et al., Deep transformer models for time series forecasting: the influenza prevalence case, arXiv: 2001.08317 (2020).
G. Zerveas et al., A transformer-based framework for multivariate time series representation learning, arXiv: 2010.02803 (2020).
M. Bayram, T. Partal and G. O. Buyukoz, Numerical methods for simulation of stochastic differential equations, Adv. Differ. Equations, 17 (2018).
S. Seabold and J. Perktold, Statsmodels: econometric and statistical modeling with python, Proc. 9th Python Sci. Conf. (2010).
I. Sutskever, O. Vinyals and Q. V. Le, Sequence to sequence learning with neural networks, Adv. Neural Inf. Process. Syst. (2014) 3104–3112.
A. Vaswani et al., Attention is all you need, Adv. Neural Inf. Process. Syst. (2017) 6000–6010.
S. M. Kazemi et al., Time2Vec: learning a vector representation of time, arXiv: 1907.05321 (2019).
D. Ding et al., Modeling extreme events in time series prediction, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2019).
R. P. Ribeiro and N. Moniz, Imbalanced regression and extreme value prediction, Mach. Learn., 109(9) (2020) 1083–1835.
D. Qi and A. J. Majda, Using machine learning to predict extreme events in complex systems, Proc. Natl. Acad. Sci. U. S. A., 117(1) (2020) 52–59.
T. Akiba et al., Optuna: a next-generation hyperparameter optimization framework, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2019).
D. P. Kingma and J. L. Ba, Adam: a method for stochastic optimization, 3rd Int. Conf. Learn. Represent. ICLR 2015-Conf. Track Proc. (2015).
A. Paszke et al., PyTorch: an imperative style, high-performance deep learning library, Adv. Neural Inf. Process. Syst. (2019) 8024–8035.
J. Myoung et al., Optimization of the computing environment to improve the speed of the modeling (WRF and CMAQ) calculation of the national air quality forecast system, J. Environ. Sci. Int., 27(8) (2018) 723–735.
S. Yu et al., New unbiased symmetric metrics for evaluation of air quality models, Atmos. Sci. Lett., 7(1) (2006) 26–34.
Y. Li and Y. Liang, Learning overparameterized neural networks via stochastic gradient descent on structured data, Adv. Neural Inf. Process. Syst. (2018).
A. Borovykh, C. W. Oosterlee and S. M. Bohté, Generalization in fully-connected neural networks for time series forecasting, J. Comput. Sci. (2019).
B. K. Yi, H. V. Jagadish and C. Faloutsos, Efficient retrieval of similar time sequences under time warping, Proc. — Int. Conf. Data Eng. (1998).
Acknowledgments
This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP) (2017R1E1A1A03070282).
Author information
Authors and Affiliations
Corresponding author
Additional information
Jongsu Kim received his B.S. (2011) in Atmospheric Science and Computer Science from Yonsei University, Seoul, Korea. He is a Ph.D. candidate at Yonsei University in the School of Mathematics and Computing. His research interests include time series forecasting using machine learning.
Changhoon Lee received his B.S. (1985) and M.S. (1987) from Seoul National University, Seoul, Korea and his Ph.D. (1993) from UC Berkeley, USA in Mechanical Engineering. He is a Professor in the Department of Computational Science & Engineering and Department of Mechanical Engineering, Yonsei University, Korea. His research interests include the fundamentals of turbulence, particle-turbulence interaction, numerical algorithms, air pollution modeling, and stochastic processes.
Rights and permissions
About this article
Cite this article
Kim, J., Lee, C. Deep particulate matter forecasting model using correntropy-induced loss. J Mech Sci Technol 35, 4045–4063 (2021). https://doi.org/10.1007/s12206-021-0817-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12206-021-0817-4