Abstract
Content delivery networks (CDNs) play a pivotal role in the modern internet infrastructure by enabling efficient content delivery across diverse geographical regions. As an essential component of CDNs, the edge caching scheme directly influences the user experience by determining the caching and eviction of content on edge servers. With the emergence of 5G technology, traditional caching schemes have faced challenges in adapting to increasingly complex and dynamic network environments. Consequently, deep reinforcement learning (DRL) offers a promising solution for intelligent zero-touch network governance. However, the black-box nature of DRL models poses challenges in understanding and making trusting decisions. In this paper, we propose an explainable reinforcement learning (XRL)-based intelligent edge service caching approach, namely XRL-SHAP-Cache, which combines DRL with an explainable artificial intelligence (XAI) technique for cache management in CDNs. Instead of focusing solely on achieving performance gains, this study introduces a novel paradigm for providing interpretable caching strategies, thereby establishing a foundation for future transparent and trustworthy edge caching solutions. Specifically, a multi-level cache scheduling framework for CDNs was formulated theoretically, with the D3QN-based caching scheme serving as the targeted interpretable model. Subsequently, by integrating Deep-SHAP into our framework, the contribution of each state input feature to the agent’s Q-value output was calculated, thereby providing valuable insights into the decision-making process. The proposed XRL-SHAP-Cache approach was evaluated through extensive experiments to demonstrate the behavior of the scheduling agent in the face of different environmental inputs. The results demonstrate its strong explainability under various real-life scenarios while maintaining superior performance compared to traditional caching schemes in terms of cache hit ratio, quality of service (QoS), and space utilization.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Phillips N A. Content Delivery Network (CDN) Market: Global Industry Trends, Share, Size, Growth, Opportunity and Forecast 2023–2028. IMARC Market Research Report. 2022
Zolfaghari B, Srivastava G, Roy S, et al. Content delivery networks. ACM Comput Surv, 2020, 53: 1–34
Nisar K, Jimson E R, Hijazi M H A, et al. A survey on the architecture, application, and security of software defined networking: challenges and open issues. Internet Things, 2020, 12: 100289
Nygren E, Sitaraman R K, Sun J. The Akamai network. SIGOPS Oper Syst Rev, 2010, 44: 2–19
Zhu G X, Lyu Z H, Jiao X, et al. Pushing AI to wireless network edge: an overview on integrated sensing, communication, and computation towards 6G. Sci China Inf Sci, 2023, 66: 130301
Hu Z, Fang C, Wang Z, et al. Many-objective optimization-based content popularity prediction for cache-assisted cloud-edge-end collaborative IoT networks. IEEE Int Things J, 2024, 11: 1190–1200
He C, Ma M, Wang P. Extract interpretability-accuracy balanced rules from artificial neural networks: a review. Neurocomputing, 2020, 387: 346–358
Rachha A, Seyam M. Explainable AI In education: current trends, challenges, and opportunities. SoutheastCon, 2023, 2023: 232–239
Kaadoud I C, Bennetot A, Mawhin B, et al. Explaining Aha! moments in artificial agents through IKE-XAI: implicit knowledge extraction for explainable AI. Neural Netw, 2022, 155: 95–118
Wu Y, Lin G, Ge J. Knowledge-powered explainable artificial intelligence for network automation toward 6G. IEEE Netw, 2022, 36: 16–23
Bacciu D, Numeroso D. Explaining deep graph networks via input perturbation. IEEE Trans Neural Netw Learn Syst, 2023, 34: 10334–10345
Du Y, Antoniadi A M, McNestry C, et al. The role of XAI in advice-taking from a clinical decision support system: a comparative user study of feature contribution-based and example-based explanations. Appl Sci, 2022, 12: 10323
Padovan P H, Martins C M, Reed C. Black is the new orange: how to determine AI liability. Artif Intell Law, 2023, 31: 133–167
Yan H, Xu X, Dai F, et al. Service caching for meteorological emergency decision-making in cloud-edge computing. In: Proceedings of IEEE International Conference on Web Services (ICWS), 2022. 120–128
Kong X, Duan G, Hou M, et al. Deep reinforcement learning-based energy-efficient edge computing for Internet of Vehicles. IEEE Trans Ind Inf, 2022, 18: 6308–6316
Wang F, Wang F, Liu J, et al. Intelligent video caching at network edge: a multi-agent deep reinforcement learning approach. In: Proceedings of IEEE Conference on Computer Communications, 2020. 2499–2508
Fang C, Xu H, Yang Y, et al. Deep-reinforcement-learning-based resource allocation for content distribution in fog radio access networks. IEEE Int Things J, 2022, 9: 16874–16883
Nikbakht R, Kahvazadeh S, Mangues-Bafalluy J. Video on demand streaming using RL-based edge caching in 5G networks. In: Proceedings of IEEE Conference on Standards for Communications and Networking (CSCN), 2022. 208
Lim D, Lee W, Kim W T, et al. DRL-OS: a deep reinforcement learning-based offloading scheduler in mobile edge computing. Sensors, 2022, 22: 9212
Zhou X, Liu Z, Guo M, et al. SACC: a size adaptive content caching algorithm in fog/edge computing using deep reinforcement learning. IEEE Trans Emerg Top Comput, 2022, 10: 1810–1820
Wells L, Bednarz T. Explainable AI and reinforcement learning — a systematic review of current approaches and trends. Front Artif Intell, 2021, 4: 550030
Vouros G A. Explainable deep reinforcement learning: state of the art and challenges. ACM Comput Surv, 2023, 55: 1–39
Zhang K, Zhang J, Xu P D, et al. Explainable AI in deep reinforcement learning models for power system emergency control. IEEE Trans Comput Soc Syst, 2021, 9: 419–427
Dassanayake P M, Anjum A, Bashir A K, et al. A deep learning based explainable control system for reconfigurable networks of edge devices. IEEE Trans Netw Sci Eng, 2021, 9: 7–19
Zhu Y, Yin X, Chen C. Extracting decision tree from trained deep reinforcement learning in traffic signal control. IEEE Trans Comput Soc Syst, 2023, 10: 1997–2007
Chen L, Hu X, Tang B, et al. Conditional DQN-based motion planning with fuzzy logic for autonomous driving. IEEE Trans Intell Transp Syst, 2022, 23: 2966–2977
Aghaeipoor F, Sabokrou M, Fernández A. Fuzzy rule-based explainer systems for deep neural networks: from local explainability to global understanding. IEEE Trans Fuzzy Syst, 2023, 31: 3069–3080
Soares E, Angelov P P, Costa B, et al. Explaining deep learning models through rule-based approximation and visualization. IEEE Trans Fuzzy Syst, 2020, 29: 2399–2407
Mereani F, Howe J M. Exact and approximate rule extraction from neural networks with Boolean features. In: Proceedings of the 11th International Joint Conference on Computational Intelligence, Vienna Austria, 2019. 424–433
Dhebar Y, Deb K, Nageshrao S, et al. Toward interpretable-AI policies using evolutionary nonlinear decision trees for discrete-action systems. IEEE Trans Cybern, 2024, 54: 50–62
Qiao L, Wang W, Lin B. Learning accurate and interpretable decision rule sets from neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, New York, 2021. 4303–4311
Singh I, Smith P J, Dmochowski P A. Optimal SNR analysis for single-user RIS systems in Ricean and Rayleigh environments. IEEE Trans Wireless Commun, 2022, 21: 9834–9849
Xu X, Tian H, Zhang X, et al. DisCOV: distributed COVID-19 detection on X-ray images with edge-cloud collaboration. IEEE Trans Serv Comput, 2022, 15: 1206–1219
Gao Z H, Chen X M, Shao X D. Robust federated learning for edge-intelligent networks. Sci China Inf Sci, 2022, 65: 132306
Yang C, Xu X, Zhou X, et al. Deep Q network-driven task offloading for efficient multimedia data analysis in edge computingassisted IoV. ACM Trans Multimedia Comput Commun Appl, 2022, 18: 1–24
Zhao H, Wang Q, Wang J, et al. Popularity-based and version-aware caching scheme at edge servers for multi-version VoD systems. IEEE Trans Circ Syst Video Technol, 2020, 31: 1234–1248
Einziger G, Eytan O, Friedman R, et al. Lightweight robust size aware cache management. ACM Trans Storage, 2022, 18: 1–23
Cho M, Kang D. ML-CLOCK: efficient page cache algorithm based on perceptron-based neural network. Electronics, 2021, 10: 2503
Araf S, Saha A S, Kazi S H, et al. UAV assisted cooperative caching on network edge using multi-agent actor-critic reinforcement learning. IEEE Trans Veh Technol, 2022, 72: 2322–2337
Hu H, Wu D, Zhou F, et al. Intelligent resource allocation for edge-cloud collaborative networks: a hybrid DDPG-D3QN approach. IEEE Trans Veh Technol, 2023, 72: 10696–10709
Yang Y, Lou K, Wang E, et al. Multi-agent reinforcement learning based file caching strategy in mobile edge computing. IEEE ACM Trans Netw, 2023, 31: 3159–3174
Zhang D, Wang W, Zhang J, et al. Novel edge caching approach based on multi-agent deep reinforcement learning for Internet of Vehicles. IEEE Trans Intell Transp Syst, 2023, 24: 8324–8338
Paleja R, Ghuy M, Arachchige N R, et al. The utility of explainable AI in ad hoc human-machine teaming. Adv Neural Inform Process Syst, 2021, 34: 610–623
Datta A, Sen S, Zick Y. Algorithmic transparency via quantitative input influence: theory and experiments with learning systems. In: Proceedings of IEEE Symposium on Security and Privacy (SP), 2016. 598–617
Lundberg S M, Lee S-I. A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17), Red Hook, 2017. 4768–4777
Shrikumar A, Greenside P, Kundaje A. Learning important features through propagating activation differences. In: Proceedings of International Conference on Machine Learning, 2017. 3145–3153
Romero F, Chaudhry G I, Goiri í, et al. Faa$T: a transparent auto-scaling cache for serverless applications. In: Proceedings of the ACM Symposium on Cloud Computing, Seattle, 2021. 122–137
Yan H, Bilal M, Xu X, et al. Edge server deployment for health monitoring with reinforcement learning in Internet of Medical Things. IEEE Trans Comput Soc Syst, 2024. doi: https://doi.org/10.1109/TCSS.2022.3161996
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (Grant No. 92267104) and Natural Science Foundation of Jiangsu Province of China (Grant No. BK20211284).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xu, X., Wu, F., Bilal, M. et al. XRL-SHAP-Cache: an explainable reinforcement learning approach for intelligent edge service caching in content delivery networks. Sci. China Inf. Sci. 67, 170303 (2024). https://doi.org/10.1007/s11432-023-3987-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-023-3987-y