Abstract
Recent years have witnessed the rapid growth of social multimedia data available over the Internet. The age of huge amount of media collection provides users facilities to share and access data, while it also demands the revolution of data management techniques, since the exponential growth of social multimedia requires more scalable, effective and robust technologies to manage and index them. The event is one of the most important cues to recall people’s past memory. The reminder value of an event makes it extremely helpful in organizing data. The study of event based analysis on social multimedia data has drawn intensive attention in research community. In this article, we provide a comprehensive survey on event based analysis over social multimedia data, including event enrichment, detection, and categorization. We introduce each paradigm and summarize related research efforts. In addition, we also suggest the emerging trends in this research area.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Mor N. Social Multimedia: highlighting opportunities for search and mining of multimedia aata in social media applications. Multimedia Tools and Applications, 2012, 56(1): 9–34
Luo J B, Dhiraj J, Yu J, Andrew G. Geotagging in multimedia and computer vision—a survey. Multimedia Tools and Applications, 2011, 51(1): 187–211
Alessandro V, Maja P, Herve B. Social signal processing: survey of an emerging domain. Image and Vision Computing, 2009, 27(12): 1743–1759
Mei T, Rui Y, Li S P, Tian Q. Multimedia search reranking: a literature survey. ACM Computing Surveys, 2014, 46(3)
Wang M, Ni B B, Hua X S, Tat-Seng C. Assistive tagging: a survey of multimedia tagging with human-computer joint exploration. ACM Computing Surveys, 2012, 44(4)
Yang Q. Three challenges in data mining. Frontiers of Computer Science in China, 2010, 4(3): 324–333
Ma H X, Qian WN, Xia F, He X F, Xu J, Zhou A Y. Towards modeling popularity of microblogs. Frontiers of Computer Science, 2013, 7(2): 171–184
Utz W, Ramesh J. Toward a common event model for multimedia applications. IEEE MultiMedia, 2007, 14(1): 19–29
Liu X L, Raphaël T, Benoit H. Finding media illustrating events. In: Proceedings of ACM International Conference on Multimedia Retrieval, 2011
Scherp A, Jain R, Kankanhalli M, Mezaris V. Modeling, detecting, and processing events in multimedia. In: Proceedings of the International Conference on Multimedia, 2010, 1739–1740
Petkos G, Papadopoulos S, Mezaris V, Troncy R, Cimiano P, Reuter T, Kompatsiaris Y. Social event detection at mediaeval: a threeyear retrospect of tasks and results. In: Proceedings of ACM ICMR 2014 Workshop on Social Events in Web Multimedia, 2014
Reuter T, Papadopoulos S, Petkos G, Mezaris V, Kompatsiaris Y, Cimiano P, de Vries C, Geva S. Social event detection at mediaeval 2013: challenges, datasets, and evaluation. In: Proceedings of the MediaEval 2013 Multimedia Benchmark Workshop, 2013
Troncy R, Malocha B, Fialho A T S. Linking events with media. In: Proceedings of the 6th International Conference on Semantic Systems, 2010
Becker H, Naaman M, Gravano L. Learning similarity metrics for event identification in social media. In: Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 2010, 291–300
Liu X L, Troncy, R Huet B. Usingsocial media to identifyevents. In Proceedings of the ACM SIGMM Workshop on Social Media. 2011, 3–8
Trabelsi C, Yahia S B. A probabilistic approach for events identification from social media RSS feeds. Database Systems for Advanced Applications, 2013, 7827: 139–152
Firan C S, Georgescu M, Nejdl W, Paiu R. Bringing order to your photos: event-driven classification of Flickr images based on social knowledge. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, 2010, 189–198
Liu X L, Huet B. Heterogeneous features and model selection for event-based media classification. In: Proceedings of International Conference on Multimedia Retrieval. 2013, 151–158
Billsus D, Pazzani M J. A hybrid user model for news story classification. In: Proceedings of the 7th International Conference on User Modeling. 1999, 99–108
Toda H, Kataoka R. A clustering method for news articles retrieval system. In: Proceedings of the International Conference on World Wide Web. 2005
Delgado D, Magalhães J, Correia N. Assisted news reading with automated illustrations. In: Proceedings of ACM Conference on Multimedia. 2010, 1647–1650
Joshi D, Wang J Z, Jia L. The story picturing engine—a system for automatic text illustration. ACM Transactions on Multimedia Computing Communications and Applications, 2006, 2(1): 68–89
Cooper M, Foote J, Girgensohn A, Wilcox L. Temporal event clustering for digital photo collections. In: Proceedings of ACM International Conference on Multimedia, 2003
Graham A, Garcia-Molina H, Paepcke A, Winograd T. Time as essence for photo browsing through personal digital libraries. In: Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries. 2002, 326–335
Jou B, Li H Z, Ellis J G, Morozoff-Abegauz D, Chang S F. Structured exploration of who, what, when, and where in heterogeneous multimedia news sources. In: Proceedings of the ACM International Conference on Multimedia. 2013, 357–360
Kim M, Xie L X, Christen P. Event diffusion patterns in social media. In: Proceedings of the International AAAI Conference onWeblogs and Social Media. 2012
Burton K, Kasch N, Soboroff I. The icwsm 2011 Spinn3r dataset. In: Proceedings of the 5th Annual Conference on Weblogs and Social Media. 2011
Mattivi R, Uijlings J, De Natale F, Sebe N. Categorization of a collection of pictures into structured events. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012
Mattivi R, Uijlings J, De Natale F G B, Sebe N. Exploitation of Time Constraints for (sub-)Event Recognition. In: Proceedings of the 2011 Joint ACM Workshop on Modeling and Representing Events. 2011, 7–12
Cooper M, Foote J, Girgensohn A, Wilcox L. Temporal event clustering for digital photo collections. ACM Transactions on Multimedia Computing, Communications, and Applications, 2005, 1(3): 269–288
Sinha P, Jain R. Extractive summarization of personal photos from life events. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2011
Kennedy L, Naaman m. Less talk, more rock: automated organization of community-contributed collections of concert videos. In: Proceedings of the 18th ACM International Conference on World Wide Web. 2009, 311–320
Diakopoulos N, Naaman M, Kivran-Swaine F. Diamonds in the rough: social media visual analytics for journalistic inquiry. In: Proceedings of 2010 IEEE Symposium on Visual Analytics Science and Technology. 2010, 115–122
Gao M Y, Hua X S, Jain R. Wonder What: real-time event determination from photos. In: Proceedings of the 20th World Wide Web Conference. 2011
Liu X L, Huet B. Event representation and visualization from social media. Lecture Notes in Computer Science, 2013, 8294: 740–749
Tang J L, Wang X F, Gao H J, Hu X, Liu H. Enriching short text representation in microblog for clustering. Frontiers of Computer Science, 2012, 6(1): 88–101
Zheng X L, Zhong Y G, Daniel Z, Wang F Y. Social influence and spread dynamics in social networks. Frontiers of Computer Science, 2012, 6(5): 611–620
Liu X L, Huet B. Automatic concept detector refinement for large-scale video semantic annotation. In: Proceedings of the 4th IEEE International Conference on Semantic Computing. 2010, 97–100
Bird S, Klein E, Loper E. Natural language processing with python. Sebastopol: O’Reilly Media, Inc., 2009
Auer S, Bizer C, Kobilarov G, Lehmann J, Ives Z. Dbpedia: a nucleus for a Web of open data. In: Proceedings of the 6th International Semantic Web Conference. 2007, 11–15
Sun C J, Guan Y. A statistical approach for content extraction from Web page. Journal of Chinese Information Processing, 18(5): 17–22, 2004
Mei T, Yang B, Yang S Q, Hua X S. Video collage: presenting a video sequence using a single image. The Visual Computer, 2009, 25(1): 39–51
Aouiche K, Lemire D, Godin R. Web 2.0 OLAP: from data cubes to tag clouds. Lecture Notes in Business Information Processing, 2009
Quack T, Leibe B, Van Gool L. World-scale mining of objects and events from community photo collections. In: Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, 2008, 47–56
Papadopoulos S, Zigkolis C, Kompatsiaris Y, Vakali A. Cluster-based landmark and event detection for tagged photo collections. IEEE Multimedia, 2011, 18(1): 52–63
Becker H, Naaman H, Gravano L. Event identification in social media. In: Proceedings of the 12th International Workshop on the Web and Databases. 2009
Petkos G, Papadopoulos S, Schinas E, Kompatsiaris Y. Graph-based multimodal clustering for social event detection in large collections of images. Lecture Notes in Computer Science, 2014, 8325: 146–158
Petkos G, Papadopoulos S, Kompatsiaris Y. Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012
Rattenbury T, Good N, Naaman M. Towards automatic extraction of event and place semantics from flickr tags. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 2007, 103
Liu Z Y, Chen X X, Sun M S. Mining the interests of chinese microbloggers via keyword extraction. Frontiers of Computer Science, 2012, 6(1): 76–87
Chen L, Abhishek R. Event detection from Flickr data through waveletbased spatial analysis. In: Proceedings of ACM conference on CIKM. 2009
Weng J S, Lee B S. Event detection in twitter. In: Proceedings of International AAAI Conference on Weblogs and Social Media. 2011
Deerwester S, Dumais S T, Furnas G W, Landauer T K, Harshman R. Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 1990, 41(6): 391–407
Hofmann T. Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1999, 50–57
Blei D M, Ng A Y, Jordan M I. Latent Dirichlet Allocation. Journal of Machine Learning Research. 2003, 3(4–5): 993–1022
Pan C C, Mitra P. Event detection with spatial latent dirichlet allocation. In: Proceeding of the 11th Annual International ACM/IEEE joint Conference on Digital Libraries. 2011, 349
Liu X L, Huet B. Social event discovery by topic inference. In: Proceeding of IEEE International Workshop on Image Analysis for Multimedia Interactive Services. 2012, 1–4
Wang X, Zhu F, Jiang J, Li S J. Real time event detection in twitter. Lecture Notes in Computer Science, 2013, 7923: 502–513
The Y W, Jordan M I, Beal M J, Blei D M. Hierarchical dirichlet processes. Journal of the American Statistical Association, 2004, 101
Cheng T, Wicks T. Event detection using Twitter: a spatio-temporal approach. PLoS ONE, 2014, 9(6): e97807
Reuter T, Cimiano P. Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, 2012
Wang Y X, Sundaram H, Xie L X. Social event detection with interaction graph modeling. In: Proceedings of the 20th ACM International Conference on Multimedia, 2012, 865–868
Brenner M, Izquierdo E. Social event detection and retrieval in collaborative photo collections. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012
Liu X L, Huet B, Troncy R. Eurecom mediaeval 2011 social event detection task. MediaEval, 2011
Li R H, Liu J Q, Yu J X, Chen H X, Kitagawa H. Co-occurrence prediction in a large location-based social network. Frontiers of Computer Science, 2013, 7(2): 185–194
Makkonen J, Ahonen-Myka H, Salmenkivi M. Simple semantics in topic detection and tracking. Information Retrieval, 2004, 3(7): 347–368
Reuter T, Cimiano P. Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012
Gu J M, Wu Y L, Hung W C, Tang C Y. Personal photo organization using event annotation. In: Proceedings of the 9th International Conference on Information, Communications and Signal Processing. 2013, 1–4
Qian S H, Zhang T Z, Xu C S. Multimodal supervised latent dirichlet allocation for event classification in social media. In: Proceedings of International Conference on Internet Multimedia Computing and Service, 2014, 152–157
Suchanek F M, Kasneci G, Weikum G. Yago: a large ontology from Wikipedia and WordNet. Web Semantics: Science, Services and Agents on the World Wide Web, 2008, 6(3): 203–217
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I H. The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter, 2009, 11(1): 10–18
Chang C C, Lin C J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 27:1–27
Breiman L, Friedman J, Olshen R, Stone C. Classification and regression trees. Monterey: Wadsworth and Brooks, 1984
Breiman L. Random forests. Machine Learning, 2001, 45: 5–32
Hays J, Efros A A. IM2GPS: estimating geographic information from a single image. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1–8
Lo H Y, Chang K W, Chen S T, Chiang S H, Ferng C S, Hsieh C J, Ko Y K, Kuo T T, Lai H C, Lin K Y, Wang C H, Yu H F, Lin C J, Lin H T, Lin S. An ensemble of three classifiers for KDD Cup 2009: expanded linear model, heterogeneous boosting, and selective naïve bayes. Journal of Machine Learning Research—Proceedings Track, 2009, 7: 57–64
Manning C D, Raghavan P, Schütze H. Introduction to information retrieval. Cambridge: Cambridge University Press, 2008
Chua T S, Tang J H, Hong R C, Li H J, Luo Z P, Zheng Y T. Nus-wide: a real-world Web image database from national university of singapore. In: Proceedings of ACM Conferrence on Image and Video Retrieval. 2009
Hebeler J, Fisher M, Blace R, Perez-Lopez A. Semantic Web programming. Indianapolis: Wiley Publishing, 2009
Petkos G, Papadopoulos S, Mezaris V, Kompatsiaris Y. Social event detection at mediaeval 2014: challenges, datasets, and evaluation. MediaEval, 2013
Over P, Awad G, Michel M, Fiscus J, Sanders G, Kraaij W, Smeaton A F, Quvenot G. TRECVID 2014—an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID. 2010, 15–17
Author information
Authors and Affiliations
Corresponding author
Additional information
Xueliang Liu received his MS and PhD from the University of Science and Technology of China, China and EURECOM, France in 2008 and 2012, respectively. He is currently with the School of Computing and Information, Hefei University of Technology. His current research interests include social media mining, multimodality modeling, and social event analysis.
Meng Wang received his BE and PhD in the Special Class for the Gifted Young and the Department of Electronic Engineering and Information Science from University of Science and Technology of China, China. He is a Professor of Hefei University of Technology, China. His current research interests include multimedia content analysis, search, mining, recommendation, and large-scale computing.
Benoit Huet received his PhD in computer science from University of York, UK. He is an assistant professor at the Multimedia Information Processing Group, EURECOM, France. His current research interests include largescale multimedia content analysis, mining and indexing, multimodal fusion, and socially-aware multimedia.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Liu, X., Wang, M. & Huet, B. Event analysis in social multimedia: a survey. Front. Comput. Sci. 10, 433–446 (2016). https://doi.org/10.1007/s11704-015-4583-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-015-4583-2