Skip to main content
SpringerLink
Log in

A Survey on Cloud Workflow Collaborative Adaptive Scheduling

  • Conference paper
  • First Online:
Advances in Computer, Communication and Computational Sciences

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1158))

Abstract

Objectively speaking, cloud workflow requires task assignment and virtual resource provisioning to work together in a collaborative manner for adaptive scheduling, thus to balance the interests of both supply and demand in the cloud service under the service level agreements. In this study, we present a survey on the current cloud workflow collaborative adaptive scheduling from the perspectives of resource provisioning and job scheduling, together with the existing cloud computing research, and look into the key problems to be solved and the future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. F. Wu, Q. Wu, Y. Tan, Workflow scheduling in cloud: a survey[J]. J. Supercomput. 71, 3373–3418 (2015)

    Article  Google Scholar 

  2. J. Zhang, S.E. Zhuge, Q.K. Wu, Efficient fault-tolerant scheduling on multiprocessor systems via replication and deallocation. Int. J. Embed. Syst. 6(2–3), 216–224 (2014)

    Google Scholar 

  3. T. Krylosova, Implementing container-based virtualization in a hybrid cloud. Metro. Ammattikorkeakoulu 1–35 (2014)

    Google Scholar 

  4. R.N. Calheiros, Ra Buyya, Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2015)

    Article  Google Scholar 

  5. G. Menegaz, The future of cloud computing: 5 predictions. http://www.thoughtsoncloud.com/future-cloud-computing-5-predictions/

  6. IDC, Virtualization and multicore innovations disrupt the worldwide server market. Document number: 206035 (2014)

    Google Scholar 

  7. R. Raju, J. Amudhavel, E. Saule, S. Anuja, A heuristic fault tolerant MapReduce framework for minimizing makespan in Hybrid Cloud Environment. in Proceedings of the Green Computing Communication and Electrical Engineering (ICGCCEE 2014) (IEEE, Piscataway, 2014)

    Google Scholar 

  8. G. Tian, C. Xiao, Z. XU et al., Hybrid scheduling strategy for multiple DAGs workflow in heterogeneous system. J. Sofw. 23(10), 2720–2734 (2012)

    Article  Google Scholar 

  9. Z. Cai, X. Li, J. Gupta, Critical path-based iterative heuristic for workflow scheduling in utility and cloud computing. in Proceedings of 11th International Conference on Service-Oriented Computing (Springer Berlin Heidelberg, Berlin, Germany, 2013), 207–221

    Google Scholar 

  10. W. Jing, Z. Wu, H. Liu, J. Dong, Fault-tolerant scheduling algorithm for precedence constrained tasks. Tsinghua Sci.Technol. 51(S1),1440–1444 (2011)

    Google Scholar 

  11. E.N. Alkhanak, S.P. Lee, S.U.R. Khan, Cost-aware challenges for workflow scheduling approaches in cloud computing environments: Taxonomy and opportunities. Fut. Gener. Comput. Syst. 50, 3–21 (2015)

    Google Scholar 

  12. Y. Zhao, Y. Li, R. Ioan, S. Lu, Lin Cui, Y. Zhang, W. Tian, R. Xue, A service framework for scientific workflow management in the cloud. IEEE Trans. Serv. Comput. 8(6), 930–944 (2015)

    Article  Google Scholar 

  13. F. Zhang, Q.M. Malluhi, T. Elsayed, S.U. Khan, K. Li, A.Y. Zomaya, CloudFlow: A data-aware programming model for cloud workflow applications on modern HPC systems. Fut Gener. Compu. Syst. 51, 98–110 (2015)

    Article  Google Scholar 

  14. M. Shojafar, S. Javanmardi, S. Abolfazli, N. Cordeschi, UGE: A joint meta-heuristic approach to cloud job scheduling algorithm using fuzzy theory and a genetic method. Cluster Comput. 18(2), 829–844 (2015)

    Article  Google Scholar 

  15. S. Abrishami, M. Naghibzadeh, D.H.J. Epema, Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Fut. Gener. Comput. Syst. 29, 158–169 (2013)

    Google Scholar 

  16. F. Zhang, J. Cao, K. Li, S.U. Khan, K. Hwang, Multi-objective scheduling of many tasks in cloud platforms. Fut. Gener. Comput. Syst. 37, 309–320 (2014)

    Article  Google Scholar 

  17. H. Guo, Z. Chen, Y. Yu, X. Chen, A communication aware DAG workflow cost optimization model and algorithm. J. Comput. Res. Develop. 52(6), 1400–1408 (2015)

    Google Scholar 

  18. T. Huang, Y. Liang, An improved simulated annealing genetic algorithm for workflow scheduling in cloud platform. Microelectron. Comput. 33(1), 42–46 (2016)

    Google Scholar 

  19. Y. Yang, X. Peng, M. Huang, J. Bian, Cloud workflow scheduling based on discrete particle swarm optimization. J. Comput. Res. Develop. 31(12), 3677–3681 (2014)

    Google Scholar 

  20. C. Szabo, Q.Z. Sheng, T. Kroeger, Y. Zhang, Y. Jian, Science in the cloud: Allocation and execution of data-intensive scientific workflows. J. Grid Comput. 12, 245–264 (2014)

    Article  Google Scholar 

  21. W. Chen, Y.C. Lee, A. Fekete, A.Y. Zomaya, Adaptive multiple-workflow scheduling with task rearrangement. J. Supercomput. 71(4), 1297–1317 (2014)

    Google Scholar 

  22. H. Shen, X. Li, Algorithm for the cloud service workflow scheduling with setup time and deadline constraints. J. Commun. 36(6), 1–10 (2015)

    Google Scholar 

  23. X. Li, W. Yang, X. Liu, H. Cheng, E. Zhu, Y. Yang, Datacenter-oriented data placement strategy of workflows in hybrid cloud. J. Soft. 27(7), 1861–1875 (2016)

    MathSciNet  Google Scholar 

  24. D. Cui, W. Ke, Z. Peng, J. Zuo, Multiple dags workflow scheduling algorithm based on reinforcement learning in cloud computing. in Proceeding of 7th International Symposium on Computational Intelligence and Intelligent Systems (Guangzhou, China, 2015), pp. 305–311

    Google Scholar 

  25. D. Cui, W. Ke, Z. Peng, J. Zuo, Cloud workflow scheduling algorithm based on reinforcement learning. Submit Int. J. High Perform. Comput. Netw. Accepted

    Google Scholar 

  26. Z. Peng, D. Cui, J. Zuo, Q. Li, Random task scheduling scheme based on reinforcement learning in cloud computing. Cluster Comput. 18(4), 1595–1607 (2015)

    Article  Google Scholar 

  27. Z. Peng, D. Cui, Q. Li, B. Xu, J. Xiong, W. Lin, A reinforcement learning-based mixed job scheduler scheme for cloud computing under SLA constraint. Submit to Soft Comput

    Google Scholar 

  28. H.M. Fard, R. Prodan, J.J.D. Barrionuevo, T. Fahringer, A multi-objective approach for workflow scheduling in heterogeneous environments. in Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (Ottawa, Canada, 2012), pp. 300–309

    Google Scholar 

  29. Y. Liu, Q. Zhao, W. Jing, Task scheduling algorithm based on dynamic priority and firefly behavior in cloud computing. Appl. Res. Comput. 32(4), 1040–1043 (2015)

    Google Scholar 

  30. W. Jing, Z. Wu, H. Liu, Y. Shu, Multiple DAGs dynamic workflow reliability scheduling algorithm in cloud computing system. J. Xidian Univ. 43(2), 92–97 (2016)

    Google Scholar 

  31. G. Tian, C. Xiao, J. Xie, Scheduling and fair cost-optimizting methods for concurrent multiple DAGs with Deadline sharing resources. Chin. J. Comput. 37(7), 1607–1619 (2014)

    Google Scholar 

  32. M.A. Rodriguez, R. Buyya, Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)

    Article  Google Scholar 

  33. Z. Chen, K. Du, Z. Zhan, Deadline constrained cloud computing resources scheduling for cost optimization based on dynamic objective genetic algorithm. in Proceedings of the 2015 IEEE Congress on Evolutionary Compution (IEEE, Sendai, Japan, 2015), pp. 708–714

    Google Scholar 

  34. Y. Wang, J. Wang, C. Wang, Y. Han, Modified scheduling algorithm for cloud workflow based on QoS. J. Northeastern Univ. (Nat. Sci.) 35(7), 939–943 (2014)

    Google Scholar 

  35. E. Byun, Y. Kee, J. Kim, Cost optimized provisioning of elastic resources for application workflows. Fut. Gener. Comput. Syst. 27, 1011–1026 (2011)

    Article  Google Scholar 

  36. E. Byun, Y. Kee, J. Kim, E. Deelman, S. Maeng, BTS: Resource capacity estimate for time-targeted science workflows. J. Paral. Distrib. Comput. 71, 848–862 (2011)

    Article  Google Scholar 

  37. Z. Peng, D. Cui, J. Zuo, Research on cloud computing resources provisioning based on reinforcement learning. Mathe. Problems Eng. 9, 1–12 (2015)

    Google Scholar 

  38. B. Xu, Z. Peng, F. Xiao, A.M. Gates, J.P. Yu, Dynamic deployment of virtual machines in cloud computing using multi-objective optimization. Soft Comput. 19(8), 2265–2273 (2015)

    Google Scholar 

  39. K. Maheshwari, E.S. Jung, J. Meng, V. Morozov, V. Vishwanath, R. Kettimuthu, Workflow performance improvement using model-based scheduling over multiple clusters and clouds. Fut. Gener. Comput. Syst. 54, 206–218 (2016)

    Article  Google Scholar 

  40. H.F.M. Reza, Y.C. Lee, A.Y. Zomaya, Randomized approximation scheme for resource allocation in hybrid-cloud environment. J Supercomput. 69(2), 576–592 (2012)

    Google Scholar 

  41. W. Wang, B. Liang, Multi-resource fair allocation in heterogeneous cloud computing systems. IEEE Trans. Parallel Distrib. Syst. 26(10), 2822–2835 (2016)

    Article  Google Scholar 

  42. L. Zuo, S. Dong, Resource scheduling methods based on deadline and cost constraint in hybrid cloud. Appl. Res. Comput. 33(8), 2315–2319 (2016)

    Google Scholar 

  43. E. Deelman, K. Vahi, M. Rynge, G. Juve, R. Mayani, R.F. da Silva, Pegasus in the cloud: science automation through workflow technologies. IEEE Internet Comput. 20(1), 70–76 (2016)

    Google Scholar 

  44. http://www.egi.eu/

  45. M.A. Vasile, F. Pop, R.I. Tutueanua, V. Cristea, J. Kołodziej, Resource-aware hybrid scheduling algorithm in heterogeneous distributed computing. Fut. Gener. Comput. Syst. 51, 61–71 (2015)

    Google Scholar 

  46. H. Jiao, J. Zhang, J. Li, J. Shi, J. Li, Cloud workflow model for collaborative design based on hybrid petri net. J. Appl. Sci. 32(6), 645–651 (2014)

    Google Scholar 

  47. M. Hussin, N.A.W.A. Hamid, K.A. Kasmiran, Improving reliability in resource management through adaptive reinforcement learning for distributed systems. J. Parallel. Distri. Comput. 75, 93–100 (2015)

    Article  Google Scholar 

  48. E. Barrett, E. Howley, J. Duggan, Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Computat.-Pract. Exp. 25(12), 1656–1674 (2013)

    Article  Google Scholar 

  49. Y. Guo, P. Lama, J. Rao, V-cache: towards flexible resource provisioning for multi-tier applications in IaaS clouds. in Proceedings of IEEE 27th International Parallel And Distributed Processing Symposium (2013), pp. 88–99

    Google Scholar 

  50. J. Rao, X. Bu, C. Xu, K. Wang, A distributed self-learning approach for elastic provisioning of virtualized cloud resources. in Proceedings of IEEE 19th International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems (2011), pp. 45–54

    Google Scholar 

  51. F. Farahnakian, T. Pahikkala, P. Liljeberg, J. Plosila, H. Tenhunen, Multi-agent Based Architecture For Dynamic VM Consolidation in Cloud Data centers (2014), pp. 111–118

    Google Scholar 

  52. M. Hussin, C.L. Young, A.Y. Zomaya, Efficient energy management using adaptive reinforcement learning-based scheduling in large-scale distributed systems. in Proceedings of International Conference on Parallel Processing (2011), pp. 385–393

    Google Scholar 

  53. B. Yang, X. Xu, F. Tan, D.H. Park, An utility-based job scheduling algorithm for Cloud computing considering reliability factor. in Proceedings of International Conference on Cloud and Service Computing (2011), pp. 95–102

    Google Scholar 

  54. X. Bu, J. Rao C.Z. Xu, A reinforcement learning approach to online web systems auto-configuration. in Proceedings of 29th IEEE International Conference on Distributed Computing Systems (2009), pp. 2–11

    Google Scholar 

Download references

Acknowledgements

The work presented in this paper was supported by National Natural Science Foundation of China (No. 61672174, 61772145). National Natural Science Foundation of China under Grants (No. 61803108). Maoming Engineering Research Center for Automation in Petro-Chemical Industry (No. 517013), and Guangdong University Student Science and Technology Innovation Cultivation Special Fund (no. pdjh2019b0326, 733409). Zhiping Peng is corresponding author.

Author information

Authors and Affiliations

  1. College of Computer and Electronic Information, Guangdong University of Petrochemical Technology, 525000, Maoming, China

    Delong Cui, Zhiping Peng, Qirui Li, Jieguang He, Lizi Zheng & Yiheng Yuan

Authors
  1. Delong Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiping Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qirui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jieguang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lizi Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yiheng Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Delong Cui .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, University of Missouri–St. Louis, Chesterfield, MO, USA

    Sanjiv K. Bhatia

  2. Computer Science Engineering Department, ABES Engineering College, Ghaziabad, Uttar Pradesh, India

    Shailesh Tiwari

  3. Shanghai Advanced Research Institute, Pudong, China

    Su Ruidan

  4. National Institute of Technology Agartala, Agartala, Tripura, India

    Munesh Chandra Trivedi

  5. Computer Science Engineering Department, Motilal Nehru National Institute of Technology, Allahabad, Uttar Pradesh, India

    K. K. Mishra

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cui, D., Peng, Z., Li, Q., He, J., Zheng, L., Yuan, Y. (2021). A Survey on Cloud Workflow Collaborative Adaptive Scheduling. In: Bhatia, S.K., Tiwari, S., Ruidan, S., Trivedi, M.C., Mishra, K.K. (eds) Advances in Computer, Communication and Computational Sciences. Advances in Intelligent Systems and Computing, vol 1158. Springer, Singapore. https://doi.org/10.1007/978-981-15-4409-5_11

Download citation

Publish with us

Policies and ethics

Search

Navigation