Abstract
In recent years energy market has changed. Consumers in many countries are free to buy energy from any of the available providers. This requires continuous reading from a huge number of energy meters to evaluate the amount of energy being bought from a particular provider. In this paper we present a fault-tolerant distributed stream processing system for continuous meter readings. The main goal of the system is to store the readings in a stream data warehouse for further analysis. We focus on modeling of the data stream intensity in order to estimate the size of buffers in a network of components composing the system. We present both the mathematical model of the intensity and the simulation results to prove the correctness of the theoretical analysis.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Wrembel, R.: On Handling the Evolution of External Data Sources in a Data Warehouse Architecture: Integrations of Data Warehousing. In: Taniar, D., Chen, L. (eds.) Integrations of Data Warehousing, Data Mining and Database Technologies, pp. 106–147 (2011)
Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., Widom, J.: Stream: The stanford stream data manager. IEEE Data Eng. Bull. 26(1), 19–26 (2003)
Gorawski, M., Chrószcz, A.: Synchronization Modeling in Stream Processing. In: Morzy, T., Härder, T., Wrembel, R. (eds.) Advances in Databases and Information Systems. AISC, vol. 186, pp. 91–102. Springer, Heidelberg (2013)
Gorawski, M., Chroszcz, A.: Optimization of operator partitions in stream data warehouse. In: Song, L.-Y., Cuzzocrea, A., Davis, K.C. (eds.) DOLAP 2011, pp. 61–66. ACM (2011)
Gorawski, M., Malczok, R.: Indexing Spatial Objects in Stream Data Warehouse. In: Nguyen, N.T., Katarzyniak, R., Chen, S.-M. (eds.) Advances in Intelligent Information and Database Systems. SCI, vol. 283, pp. 53–65. Springer, Heidelberg (2010)
Gorawski, M., Malczok, R.: Answering Range-Aggregate Queries over Objects Generating Data Streams. In: Kitagawa, H., Ishikawa, Y., Li, Q., Watanabe, C. (eds.) DASFAA 2010. LNCS, vol. 5982, pp. 436–439. Springer, Heidelberg (2010)
Gorawski, M.: Multiversion Spatio-temporal Telemetric Data Warehouse. In: Grundspenkis, J., Kirikova, M., Manolopoulos, Y., Novickis, L. (eds.) ADBIS 2009. LNCS, vol. 5968, pp. 63–70. Springer, Heidelberg (2010)
Kwiecień, A., Opielka, K.: Industrial Networks in Explosive Atmospheres. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 367–378. Springer, Heidelberg (2011)
Kwiecień, A., Stój, J.: Genius Network Communication Process Registration and Analysis. In: Kwiecień, A., Gaj, P., Stera, P. (eds.) CN 2011. CCIS, vol. 160, pp. 314–321. Springer, Heidelberg (2011)
Balazinska, M., Balakrishnan, H., Madden, S., Stonebraker, M.: Fault-Tolerance in the Borealis Distributed Stream Processing System. In: ACM SIGMOD Conf., Baltimore, MD (2005)
Madden, S., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data: ICDE. In: Proceedings of the 18th International Conference on Data Engineering, pp. 555–566. IEEE Computer Society (2002)
Gorawski, M., Malczok, R.: Distributed spatial data warehouse indexed with virtual memory aggregation tree. In: Sander, J., Nascimento, M.A. (eds.) STDBM, pp. 25–32 (2004)
Gorawski, M., Bańkowski, S., Gorawski, M.: Selection of Structures with Grid Optimization in Multiagent Data Warehouse. In: Fyfe, C., Tino, P., Charles, D., Garcia-Osorio, C., Yin, H. (eds.) IDEAL 2010. LNCS, vol. 6283, pp. 292–299. Springer, Heidelberg (2010)
Gorawski, M., Marks, P.: High efficiency of hybrid resumption in distributed data warehouses. In: DEXA Workshops, pp. 323–327. IEEE Computer Society (2005)
Gorawski, M., Marks, P.: Checkpoint-based resumption in data warehouses. In: Socha, K. (ed.) IFIP International Federation for Information Processing. Software Engineering Techniques: Design for Quality, vol. 227, pp. 313–323. Springer, Boston (2006)
Labio, W., Wiener, J.L., Garcia-Molina, H., Gorelik, V.: Efficient resumption of interrupted warehouse loads. In: Chen, W., Naughton, J.F., Bernstein, P.A. (eds.) SIGMOD Conference, pp. 46–57. ACM (2000)
Gorawski, M., Marks, P.: Fault-tolerant distributed stream processing system. In: DEXA Workshops, pp. 395–399. IEEE Computer Society (2006)
Gorawski, M., Marks, P.: Towards reliability and fault-tolerance of distributed stream processing system. In: DepCoS-RELCOMEX, pp. 246–253. IEEE Computer Society (2007)
Gorawski, M., Marks, P.: Distributed stream processing analysis in high availability context. In: ARES 2007: Proceedings of the The Second International Conference on Availability, Reliability and Security, pp. 61–68. IEEE Computer Society, Washington, DC (2007)
Gorawski, M., Marks, P.: Towards automated analysis of connections network in distributed stream processing system. In: Haritsa, J.R., Kotagiri, R., Pudi, V. (eds.) DASFAA 2008. LNCS, vol. 4947, pp. 670–677. Springer, Heidelberg (2008)
Gorawski, M., Marks, P., Gorawski, M.: Collecting data streams from a distributed radio-based measurement system. In: Haritsa, J.R., Kotagiri, R., Pudi, V. (eds.) DASFAA 2008. LNCS, vol. 4947, pp. 702–705. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gorawski, M., Marks, P., Gorawski, M. (2013). Modeling Data Stream Intensity in Distributed Stream Processing System. In: Kwiecień, A., Gaj, P., Stera, P. (eds) Computer Networks. CN 2013. Communications in Computer and Information Science, vol 370. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38865-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-38865-1_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38864-4
Online ISBN: 978-3-642-38865-1
eBook Packages: Computer ScienceComputer Science (R0)