Abstract
An adaptive dynamic load balancing algorithm based on QoS is proposed to improve the performance of load balancing in distributed file system, combining the advantages of a variety of load balancing algorithms. The new algorithm uses a tuple containing the number of files and the total file size as the QoS measure for the requested task. The master node sets a threshold for the requested task based on the QoS to filter storage nodes that meet the requirements of the task. In order to guarantee the reliability of the new algorithm, we consider the impact of CPU utilization, memory usage, disk IO occupancy rate, network bandwidth usage and hard disk usage on load balancing performance when calculating the real-time load balancing of storage nodes. The heterogeneity of the network is considered when the master node schedule task assignments to ensure the fairness of the algorithm. The comprehensive evaluation value is determined based the performance load ratio, which is calculated from the real-time load value of the storage node and a performance value after normalization. The master node assigns tasks to the storage node with the highest comprehensive evaluation value. The storage nodes provide adaptive feedback based on changes in the degree of connectivity, rather than periodic update of the load information. The actual distributed file system environment is set up on the server cluster, the performance of the new algorithm is tested through a contrast experiment. The experimental results show that the new algorithm can effectively reduce the average response time of the system, improve throughput, and enable the system load to reach a good balance.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
E. Levy, A. Silberschatz. Distributed file systems: concepts and examples [J]. ACM computing surveys, 1990, 22(4): 321–374.
T. Z. Zhao, S. B. Dong, M. Verdi, et al. Performance evaluation and relative predictive model of parallel file system [J]. Journal of software, 2011, 22(9): 2206–2221.
M. Satyanarayanan. Distributed file systems [J]. Distributed systems. Addison-Wesley and ACM Press, 1993, 821:145–154.
S. Ghemawat, H. Gobioff, S. T. Leung. The Google file system [J]. ACM SIGOPS operating systems review, 2003, 37(5): 29–43. [5]_K. Shvachko, H. Kuang, S. Radia, et al. The hadoop distributed file system [C]//2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), IEEE, 2010:1–10.
X. Liu, Q. Yu, J. Liao. FastDFS: a high performance distributed file system [J]. ICIC express letters, Part B, 2014, 5(6): 1741–1746.
R. Achar, P. S. Thilagam, N. Soans, et al. Load balancing in cloud based on live migration of virtual machines [C]//Annual IEEE India Conference (INDICON), IEEE, 2013:1–5.
D. R. Karger, M. Ruhl. Simple efficient load balancing algorithms for peer-to-peer systems [C]//Proceedings of the Sixteenth Annual ACM Symposium on Parallelism in Algorithms and Architectures, ACM, 2004:36–43.
A. M. Alakeel. A guide to dynamic load balancing in distributed computer systems [J]. International journal of computer science and information security, 2010, 10(6): 153–160.
S. Sharma, S. Singh, M. Sharma. Performance analysis of load balancing algorithms [J]. World academy of science, engineering and technology, 2008, 38(3): 269–272.
T. L. Casavant, J. G. Kuhl. A taxonomy of scheduling in general-purpose distributed computing systems [J]. IEEE transactions on software engineering, 1988, 14(2): 141–154.
A. N. Tantawi, D. Towsley. Optimal static load balancing in distributed computer systems [J]. Journal of the ACM, 1985, 32(2): 445–465.
J. Andonieh, A. Rahman. Dynamic feedback load balancing [P]. U.S. Patent Application 13/173, 995, 2011: 6–30.
T. Kunz. The influence of different workload descriptions on a heuristic load balancing scheme [J]. IEEE transactions on software engineering, 1991, 17(7): 725–730.
H. Rahmawan, Y. S. Gondokaryono. The simulation of static load balancing algorithms [C]//International Conference on Electrical Engineering and Informatics, IEEE, 2009. 2:640–645.
D. Grosu, A. T. Chronopoulos. A truthful mechanism for fair load balancing in distributed systems [C]//Second IEEE International Symposium on Network Computing and Applications, IEEE, 2003:289–296.
D. C. Devi, V. R. Uthariaraj. Load balancing in cloud computing environment using improved weighted round robin algorithm for nonpreemptive dependent tasks [J]. The scientific world journal, 2016, 2016: 3896065.
A. Roy, D. Dutta. Dynamic load balancing: improve efficiency in cloud computing [J]. International journal of emerging research in management technology, 2013, 2(4): 78–82.
N. S. Raghava, D. Singh. Comparative study on load balancing techniques in cloud computing [J]. Open journal of mobile computing and cloud computing, 2014, 1(1): 18–25.
B. S. Rajeshwari. Comprehensive study on load balancing [J]. An international journal of advanced computer technology, 2014, 3(6): 900–907.
X. Qin, H. Jiang, Y. Zhu, et al. A dynamic load balancing scheme for I/O-intensive applications in distributed systems [C]//International Conference on Parallel Processing Workshops, IEEE, 2003:79–86.
T. Alam, Z. Raza. Load balancing with random information exchanged based policy [C]//IEEE International Advance Computing Conference (IACC), IEEE, 2014:690–695.
O. A. H. Hassan, L. Ramaswamy. Message replication in unstructured peer-to-peer network [C]//International Conference on Collaborative Computing: Networking, Applications and Worksharing, IEEE, 2007:337–344.
M. X. Huang, X. L. Ye, S. P. Wei, et al. A strategy of dynamic replica creation in cloud storage [C]//1st International Workshop on Cloud Computing and Information Security. Atlantis Press, 2013.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported in part by the National Basic Research Program of China (“973” Program) (No. 2013CB329102).
Ming Wang [corresponding author] received the B.E. degree in Communication engineering from Jilin University. He is currently working toward the master’s degree in Beijing University of Posts and Telecommunications. His current research interests are in the areas of distributed systems, mobile Internet and future network.
Jianfeng Guan received his B.S. degree from Northeastern University. He received the Ph.D. degree in communications and information system from the Beijing Jiaotong University. He is an associate professor at Beijing University of Posts and Telecommunications. His main research interests focus around mobile Internet, network security and future network.
Rights and permissions
About this article
Cite this article
Wang, M., Guan, J. An adaptive dynamic feedback load balancing algorithm based on QoS in distributed file system. J. Commun. Inf. Netw. 2, 30–40 (2017). https://doi.org/10.1007/s41650-017-0029-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41650-017-0029-3