Abstract
Event detection has been an important task for a long time. When it comes to Twitter, new problems are presented. Twitter data is a huge temporal data flow with much noise and various kinds of topics. Traditional sophisticated methods with a high computational complexity aren’t designed to handle such data flow efficiently. In this paper, we propose a mixture Gaussian model for bursty word extraction in Twitter and then employ a novel time-dependent HDP model for new topic detection. Our model can grasp new events, the location and the time an event becomes bursty promptly and accurately. Experiments show the effectiveness of our model in real time event detection in Twitter.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Kulldorff, M., Mostashari, F., Duczmal, L., Yih, W.K., Kleinman, K., Platt, R.: Multivariate scan statistics for disease surveillance. Statistics in Medicine 26, 1824–1833 (2007)
Donoho, D., Jin, J.: Higher criticism for detecting sparse heterogeneous mixtures. Annals of Statistics 32(3), 962–994 (2004)
Weng, J., Lee, B.-S.: Event detection in twitter. In: ICWSM 2011, Barcelona, Spain (2011)
Diao, Q., Jiang, J., Zhu, F., Lim, E.-P.: Finding bursty topics from microblogs. In: ACL 2012, Jeju, Korea (2012)
Sakaki, T., Okazaki, M., Matsuo, Y.: Earthquake shakes Twitter users: real-time event detection by social sensors. In: WWW 2010, New York, USA (2010)
Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical Dirichlet Processes. Journal of the American Statistical Association 101(476), 1566–1581 (2006)
Kleinberg, J.: 2002 Bursty and hierarchical structure in streams. In: KDD 2002, New York, USA (2002)
Fung, G.P.C., Yu, J.X., Yu, P.S., Lu, H.: 2005 Parameter free bursty events detection in text streams. In: VLDB 2005, Trondheim, Norway (2005)
He, Q., Chang, K., Lim, E.-P.: Analyzing feature trajectories for event detection. In: SIGIR 2007, New York, USA (2007)
Chakrabarti, D., Kumar, R., Tomkins, A.: Evolutionary clustering. In: KDD 2006, Philadelphia, USA (2006), 2006
Sethuraman, J.: A constructive definition of Dirichlet priors. Statistica Sinaca 2, 639–650 (1994)
Blackwell, D., MacQueen, J.B.: Ferguson distributions via Polya urn schemes. The Annals of Statistics 1, 353–355 (1973)
Xu, T., Zhang, Z.M., Yu, P.S., Long, B.: Dirichlet process based evolutionary clustering. In: ICDM 2008, Pisa, Italy (2008)
Ren, L., Dunson, D.B., Carin, L.: The dynamic hierarchical Dirichlet process. In: ICML 2008, Helsinki, Finland (2008)
Gao, Z.J., Song, Y., Liu, S.: Tracking and Connecting Topics via Incremental Hierarchical Dirichlet Processes. In: ICDE 2011, Hannover, Germany (2011)
John, L., Andrew, M., Fernando, P.: Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In: ICML 2001, Williamstown, USA (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, X., Zhu, F., Jiang, J., Li, S. (2013). Real Time Event Detection in Twitter. In: Wang, J., Xiong, H., Ishikawa, Y., Xu, J., Zhou, J. (eds) Web-Age Information Management. WAIM 2013. Lecture Notes in Computer Science, vol 7923. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38562-9_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-38562-9_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38561-2
Online ISBN: 978-3-642-38562-9
eBook Packages: Computer ScienceComputer Science (R0)