Abstract
Today’s microblogging services such as Twitter have long outgrown their initial designs as SMS-based social networks. Instead, a massive and steadily-growing user population of more than 100 million is using Twitter for everything from capturing the mood of the country to detecting earthquakes and Internet service failures. It is unsurprising that the traditional centralized client-server architecture has not scaled with user demands, leading to server overload and significant impairment of availability. In this paper, we argue that the divergence in usage models of microblogging services can be best addressed using complementary mechanisms, one that provides reliable messages between friends, and another that delivers events from popular celebrities and media outlets to their thousands or even millions of followers. We present Cuckoo, a new microblogging system that offloads processing and bandwidth costs away from a small centralized server base while ensuring reliable message delivery. We use a 20-day Twitter availability measurement to guide our design, and trace-driven emulation of 30,000 Twitter users to evaluate our Cuckoo prototype. Compared to the centralized approach, Cuckoo achieves 30-50% server bandwidth savings and 50-60% CPU load reduction, while guaranteeing reliable message delivery.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
AFP.com. ‘Twitters’ Beat Media in Reporting China Earthquake (2008)
Buchegger, S., Schiöberg, D., Vu, L.-H., Datta, A.: PeerSoN: P2P Social Networking – Early Experiences and Insights. In: Proc. of SNS (2009)
Castro, M., Costa, M., Rowstron, A.: Debunking Some Myths about Structured and Unstructured Overlays. In: Proc. of NSDI (2005)
Cha, M., Haddadi, H., Benevenuto, F., Gummadi, K.P.: Measuring User Influence in Twitter: The Million Follower Fallacy. In: Proc. of ICWSM (2010)
CNET News. Twitter Crippled by Denial-of-Service Attack (2009)
Cutillo, L.A., Molva, R., Strufe, T.: Safebook: A Privacy Preserving Online Social Network Leveraging on Real-Life Trust. IEEE Communication Magazine 47(12), 94–101 (2009)
Eugster, P.T., Guerraoui, R., Kermarrec, A.M., Massoulié, L.: From Epidemics to Distributed Computing. IEEE Computer 37, 60–67 (2004)
Examiner.com. San Francisco Twitter Users Shocked to Lose All Their Followers (2010)
FastCompany.com. Twitter Predicts Box-Office Sales Better Than a Prediction Market (2010)
Frey, D., Guerraoui, R., Kermarrec, A.-M., Koldehofe, B., Mogensen, M., Monod, M., Quéma, V.: Heterogeneous Gossip. In: Bacon, J.M., Cooper, B.F. (eds.) Middleware 2009. LNCS, vol. 5896, pp. 42–61. Springer, Heidelberg (2009)
Ghosh, S., Korlam, G., Ganguly, N.: The Effects of Restrictions on Number of Connections in OSNs: A Case-Study on Twitter. In: Proc. of WOSN (2010)
Guardian.com. Twitter Election Predictions Are More Accurate Than YouGov (2010)
Gyarmati, L., Trinh, T.A.: Measuring User Behavior in Online Social Networks. IEEE Network Magazine, Special Issue on Online Social Networks 24(5), 26–31 (2010)
Java, A., Song, X., Finin, T., Tseng, B.: Why We Twitter: Understanding Microblogging Usage and Communities. In: Proc. of WEBKDD/SNA-KDD (2007)
Kermarrec, A.-M., Massoulié, L., Ganesh, A.: Probabilistic Reliable Dissemination in Large-Scale Systems. IEEE Transactions on Parallel and Distributed Systems 14(3), 248–258 (2003)
Krishnamurthy, B., Gill, P., Arlitt, M.: A Few Chirps about Twitter. In: Proc. of WOSN (2008)
Kwak, H., Lee, C., Park, H., Moon, S.: What is Twitter, a Social Network or a News Media? In: Proc. of WWW (2010)
Mislove, A., Post, A., Druschel, P., Gummadi, K.P.: Ostra: Leveraging Trust to Thwart Unwanted Communication. In: Proc. of NSDI (2008)
Mobilesyrup.com. Twitter CEO: 40 Percent of All Tweets Created on Mobile Devices (2011)
Motoyama, M., Meeder, B., Levchenko, K., Voelker, G.M., Savage, S.: Measuring Online Service Availability Using Twitter. In: Proc. of WOSN (2010)
New York Times. Sports Fans Break Records on Twitter (2010)
Ramasubramanian, V., Peterson, R., Sirer, E.G.: Corona: A High Performance Publish-Subscribe System for the World Wide Web. In: Proc. of NSDI (2006)
Reuters News. Twitter Snags over 100 Million Users, Eyes Money-Making (2010)
Rowstron, A., Druschel, P.: Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems. In: Liu, H. (ed.) Middleware 2001. LNCS, vol. 2218, pp. 329–351. Springer, Heidelberg (2001)
Pingdom, R.: Twitter Growing Pains Cause Lots of Downtime in 2007 (2007)
Sandler, D., Mislove, A., Post, A., Druschel, P.: FeedTree: Sharing Web Micronews with Peer-to-Peer Event Notification. In: van Renesse, R. (ed.) IPTPS 2005. LNCS, vol. 3640, pp. 141–151. Springer, Heidelberg (2005)
Sandler, D.R., Wallach, D.S.: Birds of a FETHR: Open, Decentralized Micropublishing. In: Proc. of IPTPS (2009)
Shakimov, A., Varshavsky, A., Cox, L.P., Cáceres, R.: Privacy, Cost, and Availability Tradeoffs in Decentralized OSNs. In: Proc. of WOSN (2009)
Twitter Blog. Big Goals, Big Game, Big Records (2010)
Twitter Blog. What’s Happening with Twitter? (2010)
Twitter Engineering Blog. A Perfect Storm.....of Whales (2010)
Wang, F., Xiong, Y., Liu, J.: mTreebone: A Collaborative Tree-Mesh Overlay Network for Multicast Video Streaming. IEEE Transactions on Parallel and Distributed Systems 21(3), 379–392 (2010)
Xu, T., Chen, Y., Fu, X., Hui, P.: Twittering by Cuckoo – Decentralized and Socio-Aware Online Microblogging Services. In: SIGCOMM Demo (2010)
Xu, T., Chen, Y., Jiao, L., Zhao, B.Y., Hui, P., Fu, X.: Cuckoo: Scaling Microblogging Services with Divergent Traffic Demands. Technical Report IFI-TB-2011-01, Univ. of Goettingen (2011)
Xu, T., Chen, Y., Zhao, J., Fu, X.: Cuckoo: Towards Decentralized, Socio-Aware Online Microblogging Services and Data Measurement. In: Proc. of HotPlanet (2010)
Zhao, B.Y., Huang, L., Stribling, J., Rhea, S.C., Joseph, A.D., Kubiatowicz, J.D.: Tapestry: A Resilient Global-Scale Overlay for Service Deployment. IEEE Journal on Selected Areas in Communications 22(1), 41–53 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 IFIP International Federation for Information Processing
About this paper
Cite this paper
Xu, T., Chen, Y., Jiao, L., Zhao, B.Y., Hui, P., Fu, X. (2011). Scaling Microblogging Services with Divergent Traffic Demands. In: Kon, F., Kermarrec, AM. (eds) Middleware 2011. Middleware 2011. Lecture Notes in Computer Science, vol 7049. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25821-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-25821-3_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25820-6
Online ISBN: 978-3-642-25821-3
eBook Packages: Computer ScienceComputer Science (R0)