Abstract
This paper contributes towards the mapping of the variants of Reinforcement Learning (RL) techniques to solve the key challenges of Edge Computing (EC) through broadly addressing task handling and Quality of Service (QoS) parameters. EC has bolstered ever since the advent of Industry 4.0 with computationally reliable heterogeneous mobile secured dynamic edge devices powered by an array of multifarious sensors designed on multi-edge hierarchical architectures found a strong footing on the backbone of ably equipped communication protocols to manifest their growth powered by the advent of 5G technology. However, with millions of such edge devices finding its way in a plethora of EC applications, with each having its own set of domain specific challenges, devising a suitable agent so as to sense the environment and learn from it has driven RL find its way as one of the significant tools to make the EC framework intelligent. Here we lay a good understanding of how RL has achieved noteworthy success to solve some of the pressing EC challenges, given that EC finds its use in settings of autonomous driving, content delivery, smart grid, healthcare applications and so on.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
J. Ren, H. Wang, T. Hou, S. Zheng, C. Tang, Collaborative edge computing and caching with deep reinforcement learning decision agents. IEEE Access 8, 120604–120612 (2020). https://doi.org/10.1109/ACCESS.2020.3007002
H. Zhang, T. Yu, Taxonomy of reinforcement learning algorithms, in Deep Reinforcement Learning, ed. by H. Dong, Z. Ding, S. Zhang (Springer, Singapore, 2020). https://doi.org/10.1007/978-981-15-4095-0_3
W. Shi, J. Cao, Q. Zhang, Y. Li, L. Xu, Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2016)
M. De Donno, K. Tange, N. Dragoni, Foundations and evolution of modern computing paradigms: cloud, IoT, edge, and fog. IEEE Access 7, 150936–150948 (2019). https://doi.org/10.1109/ACCESS.2019.2947652
X. Qiu, L. Liu, W. Chen, Z. Hong, Z. Zheng, Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing. IEEE Trans. Veh. Technol. 68(8), 8050–8062 (2019). https://doi.org/10.1109/TVT.2019.2924015
T.P. Lillicrap et al., Continuous Control with Deep Reinforcement Learning, Feb 2016. [Online]. Available: https://arxiv.org/abs/1509.02971
Y. Zhan, S. Guo, P. Li, J. Zhang, A deep reinforcement learning based offloading game in edge computing. IEEE Trans. Comput. 69(6), 883–893 (2020). https://doi.org/10.1109/TC.2020.2969148
X. Xiong, K. Zheng, L. Lei, L. Hou, Resource allocation based on deep reinforcement learning in IoT edge computing. IEEE J. Sel. Areas Commun. 38(6), 1133–1146 (2020). https://doi.org/10.1109/JSAC.2020.2986615
N. Din, H. Chen, D. Khan, Mobility-aware resource allocation in multi-access edge computing using deep reinforcement learning, in 2019 IEEE International Conference on Parallel and Distributed Processing with Applications, Big Data and Cloud Computing, Sustainable Computing and Communications, Social Computing and Networking (ISPA/BDCloud/SocialCom/SustainCom) (Xiamen, China, 2019), pp. 202–209. https://doi.org/10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00038
X. Liu, J. Yu, Z. Feng, Y. Gao, Multi-agent reinforcement learning for resource allocation in IoT networks with edge computing. China Commun. 17(9), 220–236 (2020). https://doi.org/10.23919/JCC.2020.09.017
C. Cho, S. Shin, H. Jeon, S. Yoon, QoS-aware workload distribution in hierarchical edge clouds: a reinforcement learning approach. IEEE Access 8, 193297–193313 (2020). https://doi.org/10.1109/ACCESS.2020.3033421
M. Tang, V.W.S. Wong, Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Trans. Mob. Comput. https://doi.org/10.1109/TMC.2020.3036871
S. Nath, J. Wu, Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems. Intell. Converged Netw. 1(2), 181–198 (2020). https://doi.org/10.23919/ICN.2020.0014
L. Ale, N. Zhang, X. Fang, X. Chen, S. Wu, L. Li, Delay-aware and energy-efficient computation offloading in mobile edge computing using deep reinforcement learning. IEEE Trans. Cogn. Commun. Netw. https://doi.org/10.1109/TCCN.2021.3066619
F.D. Vita, D. Bruneo, A. Puliafito, G. Nardini, A. Virdis, G. Stea, A deep reinforcement learning approach for data migration in multi-access edge computing, in 2018 ITU Kaleidoscope: Machine Learning for a 5G Future (ITU K) (Santa Fe, Argentina, 2018), pp. 1–8. https://doi.org/10.23919/ITU-WT.2018.8597889
R. Urimoto, Y. Fukushima, Y. Tarutani, T. Murase, T. Yokohira, A server migration method using Q-learning with dimension reduction in edge computing, in 2021 International Conference on Information Networking (ICOIN) (Jeju Island, South Korea, 2021), pp. 301–304. https://doi.org/10.1109/ICOIN50884.2021.9333965
J. Wang, J. Hu, G. Min, A.Y. Zomaya, N. Georgalas, Fast adaptive task offloading in edge computing based on meta reinforcement learning. IEEE Trans. Parall. Distrib. Syst. 32(1), 242–253 (2021). https://doi.org/10.1109/TPDS.2020.3014896
M. Yang et al., Deep reinforcement learning based green resource allocation mechanism in edge computing driven power internet of things, in 2020 International Wireless Communications and Mobile Computing (IWCMC) (Limassol, Cyprus, 2020), pp. 388–393. https://doi.org/10.1109/IWCMC48107.2020.9148169
H. Lim, J. Kim, C. Kim, G. Hwang, H. Choi, Y. Han, Federated reinforcement learning for controlling multiple rotary inverted pendulums in edge computing environments, in 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) (Fukuoka, Japan, 2020), pp. 463–464. https://doi.org/10.1109/ICAIIC48513.2020.9065233
M. Lee, C.S. Hong, Service chaining offloading decision in the EdgeAI: a deep reinforcement learning approach, in 2020 21st Asia-Pacific Network Operations and Management Symposium (APNOMS) (Daegu, South Korea, 2020), pp. 393–396. https://doi.org/10.23919/APNOMS50412.2020.9237048
J. Chen, S. Chen, Q. Wang, B. Cao, G. Feng, J. Hu, iRAF: a deep reinforcement learning approach for collaborative mobile edge computing IoT networks. IEEE Internet Things J. 6(4), 7011–7024 (2019). https://doi.org/10.1109/JIOT.2019.2913162
T. Alfakih, M.M. Hassan, A. Gumaei, C. Savaglio, G. Fortino, Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA. IEEE Access 8, 54074–54084 (2020). https://doi.org/10.1109/ACCESS.2020.2981434
S. Park, Y. Kang, Y. Tian, J. Kim, Fast and reliable offloading via deep reinforcement learning for mobile edge video computing, in 2020 International Conference on Information Networking (ICOIN) (Barcelona, Spain, 2020), pp. 10–12. https://doi.org/10.1109/ICOIN48656.2020.9016591
X. Chen, G. Liu, Joint optimization of task offloading and resource allocation via deep reinforcement learning for augmented reality in mobile edge network, in 2020 IEEE International Conference on Edge Computing (EDGE) (Beijing, China, 2020), pp. 76–82. https://doi.org/10.1109/EDGE50951.2020.00019
X. Chen, G. Liu, Energy-efficient task offloading and resource allocation via deep reinforcement learning for augmented reality in mobile edge networks. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2021.3050804
I. Khan, X. Tao, G.M.S. Rahman, W.U. Rehman, T. Salam, Advanced energy-efficient computation offloading using deep reinforcement learning in MTC edge computing. IEEE Access 8, 82867–82875 (2020). https://doi.org/10.1109/ACCESS.2020.2991057
N. Khumalo, O. Oyerinde, L. Mfupe, Reinforcement learning-based computation resource allocation scheme for 5G fog-radio access network, in 2020 Fifth International Conference on Fog and Mobile Edge Computing (FMEC) (Paris, France, 2020), pp. 353–355. https://doi.org/10.1109/FMEC49853.2020.9144787
J. Zou, T. Hao, C. Yu, H. Jin, A3C-DO: a regional resource scheduling framework based on deep reinforcement learning in edge scenario. IEEE Trans. Comput. 70(2), 228–239 (2021). https://doi.org/10.1109/TC.2020.2987567
L. Xiao, X. Lu, T. Xu, X. Wan, W. Ji, Y. Zhang, Reinforcement learning-based mobile offloading for edge computing against jamming and interference. IEEE Trans. Commun. 68(10), 6114–6126 (2020). https://doi.org/10.1109/TCOMM.2020.3007742
Z. Cao, P. Zhou, R. Li, S. Huang, D. Wu, Multiagent deep reinforcement learning for joint multichannel access and task offloading of mobile-edge computing in industry 4.0. IEEE Internet Things J. 7(7), 6201–6213 (2020). https://doi.org/10.1109/JIOT.2020.2968951
Y. Li, F. Qi, Z. Wang, X. Yu, S. Shao, Distributed edge computing offloading algorithm based on deep reinforcement learning. IEEE Access 8, 85204–85215 (2020). https://doi.org/10.1109/ACCESS.2020.2991773
Q. Liu, L. Cheng, T. Ozcelebi, J. Murphy, J. Lukkien, Deep reinforcement learning for IoT network dynamic clustering in edge computing, in 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) (Larnaca, Cyprus, 2019), pp. 600–603. https://doi.org/10.1109/CCGRID.2019.00077
Q. Guo, R. Huo, H. Meng, E. Xinhua, J. Liu, T. Huang, Research on reinforcement learning-based dynamic power management for edge data center, in 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS) (Beijing, China, 2018), pp. 865–868. https://doi.org/10.1109/ICSESS.2018.8663880
T. Yang, Y. Hu, M.C. Gursoy, A. Schmeink, R. Mathar, Deep reinforcement learning based resource allocation in low latency edge computing networks, in 2018 15th International Symposium on Wireless Communication Systems (ISWCS) (Lisbon, Portugal, 2018), pp. 1–5. https://doi.org/10.1109/ISWCS.2018.8491089
Y. Liu, H. Yu, S. Xie, Y. Zhang, Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks. IEEE Trans. Veh. Technol. 68(11), 11158–11168 (2019). https://doi.org/10.1109/TVT.2019.2935450
K. Wang, X. Wang, X. Liu, A. Jolfaei, Task offloading strategy based on reinforcement learning computing in edge computing architecture of internet of vehicles. IEEE Access 8, 173779–173789 (2020). https://doi.org/10.1109/ACCESS.2020.3023939
Z. Xue et al., A resource-constrained and privacy-preserving edge computing enabled clinical decision system: a federated reinforcement learning approach. IEEE Int. Things J. https://doi.org/10.1109/JIOT.2021.3057653
K. Kim, Y. Hong, Industrial general reinforcement learning control framework system based on intelligent edge, in 2020 22nd International Conference on Advanced Communication Technology (ICACT) (Phoenix Park, South Korea, 2020), pp. 414–418. https://doi.org/10.23919/ICACT48636.2020.9061542
F. Xu, F. Yang, C. Zhao, S. Wu, Deep reinforcement learning based joint edge resource management in maritime network. China Commun. 17(5), 211–222 (2020). https://doi.org/10.23919/JCC.2020.05.016
L. Gu, D. Zeng, W. Li, S. Guo, A. Zomaya, H. Jin, Deep reinforcement learning based VNF management in geo-distributed edge computing, in 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) (Dallas, TX, USA, 2019), pp. 934–943. https://doi.org/10.1109/ICDCS.2019.00097
K. Zhang, J. Cao, H. Liu, S. Maharjan, Y. Zhang, Deep reinforcement learning for social-aware edge computing and caching in urban informatics. IEEE Trans. Industr. Inf. 16(8), 5467–5477 (2020). https://doi.org/10.1109/TII.2019.2953189
B. Hu, J. Li, An edge computing framework for powertrain control system optimization of intelligent and connected vehicles based on curiosity-driven deep reinforcement learning. IEEE Trans. Ind. Electron. https://doi.org/10.1109/TIE.2020.3007100
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ray, A., Chakrabarti, A. (2023). Towards Efficient Edge Computing Through Adoption of Reinforcement Learning Strategies: A Review. In: Goswami, S., Barara, I.S., Goje, A., Mohan, C., Bruckstein, A.M. (eds) Data Management, Analytics and Innovation. ICDMAI 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 137. Springer, Singapore. https://doi.org/10.1007/978-981-19-2600-6_17
Download citation
DOI: https://doi.org/10.1007/978-981-19-2600-6_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-2599-3
Online ISBN: 978-981-19-2600-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)