Abstract
Over the last several years, a great deal of progress has been made in the area of stream-processing engines (SPEs). Three basic tenets distinguish SPEs from current data processing engines. First, they must support primitives for streaming applications. Unlike Online Transaction Processing (OLTP), which processes messages in isolation, streaming applications entail time series operations on streams of messages. Second, streaming applications entail a real-time component. If one is content to see an answer later, then one can store incoming messages in a data warehouse and run a historical query on the warehouse to find information of interest. This tactic does not work if the answer must be constructed in real time. The need for real-time answers also dictates a fundamentally different storage architecture. DBMSs universally store and index data records before making them available for query activity. Such outbound processing, where data are stored before being processed, cannot deliver real-time latency, as required by SPEs. To meet more stringent latency requirements, SPEs must adopt an alternate model, which we refer to as “inbound processing”, where query processing is performed directly on incoming messages before (or instead of) storing them. Lastly, an SPE must have capabilities to gracefully deal with spikes in message load. Incoming traffic is usually bursty, and it is desirable to selectively degrade the performance of the applications running on an SPE. The Aurora stream-processing engine, motivated by these three tenets, is currently operational, has been used to build various application systems, and has been transferred to the commercial domain. Borealis is a distributed stream-processing system that inherits core stream-processing functionality from Aurora and enriches it with distribution functionality, in order to provide advanced capabilities that are commonly required by newly emerging stream-processing applications.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
A guide for hot lane development: a US department of transportation federal highway administration. http://www.itsdocs.fhwa.dot.gov/JPODOCS/REPTSTE/13668.html
D. Abadi, Y. Ahmad, H. Balakrishnan, M. Balazinska, U. Cetintemel, M. Cherniack, J.-H. Hwang, J. Janotti, W. Lindner, S. Madden, A. Rasin, M. Stonebraker, N. Tatbul, Y. Xing, S. Zdonik, The design of the Borealis stream processing engine. Technical report CS-04-08, Department of Computer Science, Brown University (2004)
D. Abadi, Y. Ahmad, H. Balakrishnan, M. Balazinska, U. Cetintemel, M. Cherniack, J.-H. Hwang, W. Lindner, A. Maskey, A. Rasin, E. Rvvkina, N. Tatbul, Y. Xing, S. Zdonik, The design of the Borealis stream processing engine, in CIDR Conference (2005)
D. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, C. Erwin, E. Galvez, M. Hatoun, J. Hwang, A. Maskey, A. Rasin, A. Singer, M. Stonebraker, N. Tatbul, Y. Xing, R. Yan, S. Zdonik, Aurora: a data stream management system (demo description), in ACM SIGMOD Conference (2003)
D. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, S.Z. Aurora, A new model and architecture for data stream management. VLDB J. 12(2) (2003)
Y. Ahmad, B. Berg, U. Çetintemel, M. Humphrey, J. Hwang, A. Jhingran, A. Maskey, O. Pappaemmanouil, A. Rasin, N. Tatbul, W. Xing, Y. Xing, S. Zdonik, Distributed operation in the Borealis stream processing engine (demo description), in ACM SIGMOD Conference (2005)
A. Arasu, M. Cherniack, E.F. Galvez, D. Maier, A. Maskey, E. Ryvkina, M. Stonebraker, R. Tibbetts, Linear road: a stream data management benchmark, in VLDB (2004), pp. 480–491
M. Balazinska, H. Balakrishnan, M. Stonebraker, Contract-based load management in federated distributed systems, in NSDI Symposium (2004)
D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, G. Seidman, M. Stonebraker, N. Tatbul, S. Zdonik, Monitoring streams—a new class of data management applications, in VLDB Conference, Hong Kong, China (2002)
D. Carney, U. Çetintemel, A. Rasin, S. Zdonik, M. Cherniack, M. Stonebraker, Operator scheduling in a data stream manager, in VLDB Conference, Berlin, Germany (2003)
S. Chandrasekaran, A. Deshpande, M. Franklin, J. Hellerstein, W. Hong, S. Krishnamurthy, S. Madden, V. Raman, F. Reiss, M. Shah, TelegraphCQ: continuous dataflow processing for an uncertain world, in CIDR Conference (2003)
M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Çetintemel, Y. Xing, S. Zdonik, Scalable distributed stream processing, in CIDR Conference, Asilomar, CA (2003)
Congestion pricing: a report from intelligent transportation systems (ITS). http://www.path.berkeley.edu/leap/TTM/DemandManage/pricing.html
D. DeWitt, J. Naughton, D. Schneider, An evaluation of non-equijoin algorithms, in VLDB Conference, Barcelona, Catalonia, Spain (1991)
J. Hwang, M. Balazinska, A. Rasin, U. Çetintemel, M. Stonebraker, S. Zdonik, A comparison of stream-oriented high-availability algorithms. Technical report CS-03-17, Department of Computer Science, Brown University (2003)
A. Lerner, D. Shasha, AQuery: query language for ordered data, optimization techniques, and experiments, in VLDB Conference, Berlin, Germany (2003)
R. Motwani, J. Widom, A. Arasu, B. Babcock, S. Babu, M. Datar, G. Manku, C. Olston, J. Rosenstein, R. Varma, Query processing, approximation, and resource management in a data stream management system, in CIDR Conference (2003)
R.W. Poole, Hot lanes prompted by federal program. http://www.rppi.org/federalhotlanes.html
P. Seshadri, M. Livny, R. Ramakrishnan, SEQ: a model for sequence databases, in IEEE ICDE Conference, Taipei, Taiwan (1995)
StreamBase incorporated. http://www.streambase.com/
N. Tatbul, U. Çetintemel, S. Zdonik, M. Cherniack, M. Stonebraker, Load shedding in a data stream manager, in VLDB Conference, Berlin, Germany (2003)
The Borealis project web site. http://www.cs.brown.edu/research/borealis
The MITRE corporation. http://www.mitre.org/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Çetintemel, U. et al. (2016). The Aurora and Borealis Stream Processing Engines. In: Garofalakis, M., Gehrke, J., Rastogi, R. (eds) Data Stream Management. Data-Centric Systems and Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28608-0_17
Download citation
DOI: https://doi.org/10.1007/978-3-540-28608-0_17
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28607-3
Online ISBN: 978-3-540-28608-0
eBook Packages: Computer ScienceComputer Science (R0)