Abstract
Smart grid communication system carries a large number of business data of power grid terminals. With the wide access of new energy power, after the power grid is connected to renewable energy power, the data services required by the dispatching, control, management and other services of microgrid system need to be processed through the distribution network communication system. How to dynamically, intelligently and adaptively adjust the distribution of data flow service transmission and make each node work orderly is a new requirement of intelligent distributed data processing. This paper expounds the latest technologies from two aspects of transmission and calculation in the process of data stream processing, mainly including: data stream real-time transmission technology, service stream real-time processing system architecture, channel multiplexing processing method based on stream density, service stream real-time processing based on container technology, and introduces the application of these technologies in power grid system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Babcock, B., Babu, S., Datar, M.: Models and issues in data stream systems. In: Proceedings of the 21st ACM SIGACT SIGMOD-SIGART, pp. 1–16 (2002)
Madden, S., Franklin, M.J.: Fording the stream: an architecture for queries over streaming sensor data. In: Proceedings of the 18th International Conference on Data Engineering, pp. 555–566. Morgan Kaufmann Publishers, San Jose (2002)
Zeng, Y.F., Yang, X.J.: The optimized organization method of stream on image stream processor. J. Comput. Sci. 31(7), 1092–1100 (2008)
Kuo, K., Rabbah, R.M., Amarasinghe, S.A.: Productive Programming Environment for Stream Computing. Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology
Babu, S., Widom, J.: Continuous queries over data streams. SIGMOD Rec. 30(3), 109–120 (2001)
Arasu, A., Babcock, B.: Stream: The Stanford Stream Data Manager (Demonstrate ion Description), San Diego, CA, United States, p. 665 (2003)
Chandrasekaran, S., Franklin, M.J.: Streaming queries over streaming data. In: Proceedings of the International Conference on VLDB, pp. 203–214 (2002)
Terry, D., Goldberg, D., Nichols, D.: Continuous queries over append-only databases. In: SIGMOD (1992), pp. 321–330, June 1992
Shixuan, S.A., Wang, S.: Introduction to Database System, pp. 220–267. Higher Education Press (2000)
Tatbul, N., Cetintemel, U., Zdonik, S.B.: Load shedding in a data stream manager. In: Proceedings of the 29th International Conference on Very Large Data Bases, Berlin Germany, pp. 309–320, September 2003
Xiong, J.: Research and Application of Key Technologies of Distributed Stream Processing. University of Electronic Science and Technology, Chengdu (2017)
Qi, J., Qu, Z., Lou, J., et al.: A kind of attribute entity recognition algorithm based on Hadoop for power big data. Power Syst. Protect. Control 44(24), 52–57 (2016)
Wang, H., Fu, Y.: Heterogeneous network convergence-research development status and existing problems. Data Commun. 2, 18–21 (2012)
Gu, X.: Research and Implementation of Key Technologies of Distributed Streaming Computing Framework. Beijing University of Posts and Telecommunications, Beijing (2012)
Jiang, D., Zheng, H.: Research status and developing prospect of DC distribution network. Autom. Electr. Power Syst. 36(8), 98–104 (2012)
Chen, Z.: Data Stream Clustering Analysis and Anomaly Detection Algorithm. Fudan University, Shanghai (2009)
Liu, K., Sheng, W., Zhang, D., et al.: Big data application requirements and scenario analysis in smart distribution network. Proc. CSEE 35(2), 287–293 (2015)
Yang, X., Zeng, L., Yu, D.: The optimized approaches of organizing streams in imagine processor. Chin. J. Comput. 31(7), 1092–1100 (2008)
Yang, Y., Han, Z., Yang, L.: Survey on key technology and application development for data streams. Comput. Appl. Res. 11, 60–63 (2005)
Lu, J.: Introduction to Distributed Systems and Cloud Computing. Tsinghua University Press Xinhua News Agency, Beijing (2013)
Zhu, F., He, Y.: Theory and Design of Scheduling Algorithm in Parallel Distributed Computing. Wuhan University Press, Wu Han (2003)
Yu, Y., Lv, Z., Qi, G.: Research distributed large-scale time series data management platform. Power Syst. Protect. Control 44(17), 165–169 (2016)
Chang, G., Hao, J., Liu, B., et al.: Research and development of intelligent and classified collection system for electric power dispatching and control information. Power Syst. Protect. Control 43(6), 115–120 (2015)
Arasu, A., Babu, S., Widom, J.: The CQL continuous query language: semantic foundations and query execution. VLDB J. 15(2), 121–142 (2006)
Cortes, C., Fisher, K., Pregibon, D., et al.: Hancock: a language for analyzing transactional data streams. ACM Trans. Program. Lang. Syst. (TOPLAS) 26(2), 301–338 (2004)
Zhang, T., Liang, S., Gu, J.: Overview of the distribution and utilization big data application. Electr. Meas. Instrum. 54(2), 92–99 (2017)
Zhang, X., Yang, G., Zhao, G.: Research on automatic diagnosis and analysis technology of distribution network status based on global large power data. Electr. Meas. Instrum. 56(16), 111–115 (2019)
Acknowledgements
This paper is supported by the State Grid Anhui Electric Power Co., Ltd. Science and technology project “Research on real-time transparent access technology to improve the perception ability of power grid operation state”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, T., Zhang, H., Ma, J., Shen, X. (2023). Distributed Multi-source Service Data Stream Processing Technology and Application in Power Grid Dispatching System. In: Tang, L.C., Wang, H. (eds) Big Data Management and Analysis for Cyber Physical Systems. BDET 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 150. Springer, Cham. https://doi.org/10.1007/978-3-031-17548-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-17548-0_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17547-3
Online ISBN: 978-3-031-17548-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)