Abstract
The core function of Mapreduce is to integrate the business logic code written by the user and the default components into a complete distributed operation program and run concurrently on a hadoop cluster. MapReduce is a set of software framework, which includes two stages: Map and Reduce. It can be used to partition the massive data, decompose the task and aggregate the results, so as to complete the parallel processing of the massive data. MapReduce’s principle of work is actually the data - processing method. This article describes the analysis and application of MapReduce architecture and working principle in detail.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li W, Zhao H, Zhang Y, Wang Y (2013) Research on massive data mining based on MapReduce. Comput Eng Appl 20:100–110 (in Chinese)
Li B, Liu L (2012) Web log mining based on MapReduce. Comput Eng Appl (22):122–123 (in Chinese)
Li J, Cui J (2011) MapReduce parallel programming model. J Electron (11):520–526 (in Chinese)
Xin J, Cui Z (2011) Deep web data source discovery method based on MapReduce virtual machine. J Commun (07):320–330 (in Chinese)
Cheng M, Chen H (2011) Web log mining based on Hadoop. Comput Eng (11):108–111
Wang X, Wang Y, Zhu H (2012) Energy-efficient task scheduling model based on MapReduce for cloud computing using genetic algorithm. J Comput (12):522–526
Zhang G, Huang M, Ma L (2015) MapReduce simulator design for cloud computing environment. J Xinyang Normal Univ (Nat Sci Edn) 8(03):100–108 (in Chinese)
Xu H, Zhang R (2016) Novel approach of semantic annotation by fuzzy ontology based on variable precision rough set and concept lattice. Int J Hybrid Inf Technol 9(4):25–40
Jiang Y, Zhao Z (2015) Optimization of sorting algorithm based on MapReduce model. Comput Sci Explor (04):38–42 (in Chinese)
Jin X, Liu B (2016) Based on MapReduce location service optimization application. Inf Res 8(04):55–59
Acknowledgements
This paper is supported by Henan key Laboratory for Big Data Processing & Analytics of Electronic Commerce, and also supported by the science and technology research major project of Henan province Education Department (17B520026), Key scientific research projects in Henan province universities (17A880020, 15A120012), and the Science and Technology Opening up Cooperation project of Henan Province (No. 172106000077).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, H., Fan, G., Li, K. (2020). Analysis and Application of Mapreduce Architecture and Working Principle. In: Xu, Z., Choo, KK., Dehghantanha, A., Parizi, R., Hammoudeh, M. (eds) Cyber Security Intelligence and Analytics. CSIA 2019. Advances in Intelligent Systems and Computing, vol 928. Springer, Cham. https://doi.org/10.1007/978-3-030-15235-2_127
Download citation
DOI: https://doi.org/10.1007/978-3-030-15235-2_127
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-15234-5
Online ISBN: 978-3-030-15235-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)