Abstract
Most of the grid projects are characterized by accessing huge volumes of data. For supporting this feature, different data services have arisen in the “grid” world. One of the most successful initiatives in that field is GridFTP, a high-performance transfer protocol, based on FTP but optimized for wide area networks. Although GridFTP provides reasonably good performance, GridFTP servers keep constituting a bottleneck for data-intensive applications.
One of the most important modules of a GridFTP server is the Data Storage Interface (DSI), which specifies how to read and write to the storage system, allowing the server to transform the data. With the aim of improving the performance of the GridFTP server, we have designed a new DSI, based on MAPFS, a parallel file system. This paper describes this new DSI and its evaluation, showing the advantages of dealing data through this optimized GridFTP server.
An erratum to this chapter can be found at http://dx.doi.org/10.1007/11914952_55.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Allcock, W., Bester, J., Bresnahan, J., Chervenak, A., Liming, L., Meder, S., Tuecke, S.: GridFTP Protocol Specification. Web Page (September 2002)
Allcock, W., Bresnahan, J., Kettimuthu, R., Link, M.: The globus striped gridftp framework and server. In: SC 2005: Proceedings of the 2005 ACM/IEEE conference on Supercomputing, Washington, DC, USA, p. 54. IEEE Computer Society, Los Alamitos (2005)
Baru, C.K., Moore, R.W., Rajasekar, A., Wan, M.: The sdsc storage resource broker. In: MacKay, S.A., Johnson, J.H. (eds.) CASCON, p. 5. IBM (1998)
The Castor Project, http://www.castor.org
Chandra, P.R., Fisher, A., Kosak, C., Ng, T., Steenkiste, P., Takahashi, E., Zhang, H.: Darwin: Customizable resource management for value-added network services. In: ICNP, pp. 177–188 (1998)
Chiu, K., Govindaraju, M., Bramley, R.: Investigating the limits of soap performance for scientific computing. In: HPDC 2002: Proceedings of the 11 th IEEE International Symposium on High Performance Distributed Computing HPDC-11 2002 (HPDC 2002), Washington, DC, USA, p. 246. IEEE Computer Society, Los Alamitos (2002)
Foster, I., Kesselman, C., Nick, J.M., Tuecke, S.: The physiology of the grid: An open grid services architecture for distributed systems integration (January 2002), Published online at: http://www.globus.org/research/papers/ogsa.pdf
The Globus Project, http://www.globus.org
Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message-Passing Interface. MIT Press, Cambridge (1994)
HPSS - high performance storage system, http://www.hpss-collaboration.org
Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S.E., Eaton, P.R., Geels, D., Gummadi, R., Rhea, S.C., Weatherspoon, H., Weimer, W., Wells, C., Zhao, B.Y.: Oceanstore: An architecture for global-scale persistent storage. In: ASPLOS, pp. 190–201 (2000)
Liefke, H., Suciu, D.: Xmill: An efficient compressor for xml data. In: Chen, W., Naughton, J.F., Bernstein, P.A. (eds.) SIGMOD Conference, pp. 153–164. ACM, New York (2000)
Mpi Forum, http://www.mpi-forum.org/docs/docs.html
Pérez, M.S., Carretero, J., García, J.M.P.F., Robles, V.: MAPFS: A flexible multiagent parallel file system for clusters. Future Generation Computer Systems 22(5), 620–632 (2006)
Pérez, M.S., Carretero, J., García, F., Peña Sánchez, J.M., Robles, V.: A flexible multiagent parallel file system for clusters. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J., Zomaya, A.Y. (eds.) ICCS 2003. LNCS, vol. 2660, pp. 248–256. Springer, Heidelberg (2003)
Pérez, M.S., García, F., Carretero, J.: A new multiagent based architecture for high performance I/O in clusters. In: ICPP Workshops, pp. 201–206. IEEE Computer Society, Los Alamitos (2001)
Pérez, M.S., Sánchez, A., Peña, J.M., Robles, V.: A new formalism for dynamic reconfiguration of data servers in a cluster. J. Parallel Distrib. Comput. 65(10), 1134–1145 (2005)
Shoshany, A., et al.: SRM Interface Specification v.2.1, http://sdm.lbl.gov/srm-wg/doc/srm.spec.v2.1.final.pdf
Shoshany, A., et al.: SRM Joint Design v.1.0, http://sdm.lbl.gov/srm-wg/doc/srm.v1.0.pdf
Gt 4.0 gridftp: Storage resource broker (SRB), http://www.globus.org/toolkit/docs/4.0/data/gridftp/gridftp_srb.html
Sundaresan, N., Moussa, R.: Algorithms and programming models for efficient representation of xml for internet applications. Computer Networks 39(5), 681–697 (2002)
Tierney, B., Lee, J., Crowley, B., Holding, M., Hylton, J., Drake, F.L.: A network-aware distributed storage cache for data intensive environments. In: HPDC (1999)
Vilett, C.: Moore’s law vs. storage improvements vs. optical improvements. Scientific American (January 2001)
W3C. Soap message transmission optimization mechanism (January 2005), http://www.w3.org/tr/soap12-mtom
Watson, R.W.: High performance storage system scalability: Architecture, implementation and experience. In: MSST, pp. 145–159. IEEE Computer Society, Los Alamitos (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sánchez, A., Pérez, M.S., Gueant, P., Montes, J., Herrero, P. (2006). A Parallel Data Storage Interface to GridFTP. In: Meersman, R., Tari, Z. (eds) On the Move to Meaningful Internet Systems 2006: CoopIS, DOA, GADA, and ODBASE. OTM 2006. Lecture Notes in Computer Science, vol 4276. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11914952_10
Download citation
DOI: https://doi.org/10.1007/11914952_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-48274-1
Online ISBN: 978-3-540-48283-3
eBook Packages: Computer ScienceComputer Science (R0)