Abstract
Distributed storage systems (e.g. clustered filesystems - HDFS, GPFS and Object Stores - Openstack swift ) often partition sequential data across storage systems for performance ( data striping) or protection (Erasure-Coding) . This partitioning leads to logically correlated data being stored on different physical storage devices, which operate autonomously. This un-coordinated operation may lead to inefficient caching, where different devices may cache segments that belong to different working sets. From an application perspective, caching is effective only if all segments needed by it at a given point in time are cached and a single missing segment may lead to high application latency. In this work, we present C2P: a middleware for co-operative caching in distributed storage. C2P uses an event-based architecture to co-ordinate caching across the storage devices and ensures that all devices cache correlated segments. We have implemented C2P as a caching middleware for hosted Openstack Swift Object Store. Our experiments show 4-6% improved cache hit and 3-5% reduced disk IO with minimal resource overheads.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Adya, A., Castro, M., Liskov, B., Maheshwari, U., Shrira, L.: Fragment reconstruction: Providing global cache coherence in a transactional storage system. In: Proceedings of the 17th International Conference on Distributed Computing Systems, pp. 2–11. IEEE (1997)
Amazon: Amazon S3, http://aws.amazon.com/s3/
Bloemer, J., Kalfane, M., Karp, R., Karpinski, M., Luby, M., Zuckerman, D.: An xor-based erasure-resilient coding scheme (1995)
Cabrera, L.F., Long, D.: Using data striping in a local area network (1992)
Cabrera, L.F., Long, D.D.E.: Swift: Using distributed disk striping to provide high i/o data rates. Computing Systems 4(4), 405–436 (1991)
Chi, C.H., Lau, S.: Data prefetching with co-operative caching. In: 5th International Conference on High Performance Computing, HIPC 1998, pp. 25–32. IEEE (1998)
Clarke, K.J., Gittins, R., McPolin, S., Rang, A.: Distributed storage cache coherency system and method, US Patent 7,017,012 (March 21, 2006)
Cuenca-Acuna, F.M., Nguyen, T.D.: Cooperative caching middleware for cluster-based servers. In: Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing, pp. 303–314. IEEE (2001)
Fitzpatrick, B.: Distributed caching with memcached. Linux Journal 2004(124), 5 (2004)
Fuse: Filesystem in Userspace, http://fuse.sourceforge.net/
Google: Google Cloud Storage, http://cloud.google.com/Storage
Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., Yekhanin, S., et al.: Erasure coding in windows azure storage. In: USENIX ATC, vol. 12 (2012)
IBM: Introduction to GPFS 3.5 - IBM. RedBook (2012)
Isilon, E.: EMC Isilon OneFS: A Technical Overview. White paper (2013)
LACIE: RAID Technology. White paper
Laoutaris, N., Smaragdakis, G., Bestavros, A., Matta, I., Stavrakakis, I.: Distributed selfish caching. IEEE Transactions on Parallel and Distributed Systems 18(10), 1361–1376 (2007)
Luster: Lustre Filesystem, http://wiki.lustre.org
Microsoft: Microsoft Azure, http://azure.microsoft.com
Nishtala, R., Fugal, H., Grimm, S., Kwiatkowski, M., Lee, H., Li, H.C., McElroy, R., Paleczny, M., Peek, D., Saab, P.: et al.: Scaling memcache at facebook. In: Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation, pp. 385–398. USENIX Association (2013)
Openstack: Openstack Swift, http://swift.openstack.org
Openstack: Swiftbench. https://launchpad.net/swift-bench
RabbitMQ: RabbitMQ., https://www.rabbitmq.com/
Sarkar, P., Hartman, J.H.: Hint-based cooperative caching. ACM Transactions on Computer Systems (TOCS) 18(4), 387–419 (2000)
Weatherspoon, H., Kubiatowicz, J.D.: Erasure coding vs. replication: A quantitative comparison. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, pp. 328–337. Springer, Heidelberg (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nadgowda, S.J., Sreenivas, R.C., Gupta, S., Gupta, N., Verma, A. (2014). C2P: Co-operative Caching in Distributed Storage Systems. In: Franch, X., Ghose, A.K., Lewis, G.A., Bhiri, S. (eds) Service-Oriented Computing. ICSOC 2014. Lecture Notes in Computer Science, vol 8831. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45391-9_15
Download citation
DOI: https://doi.org/10.1007/978-3-662-45391-9_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-45390-2
Online ISBN: 978-3-662-45391-9
eBook Packages: Computer ScienceComputer Science (R0)