Abstract
Autonomous Traffic Management (ATM) is a growing field of Intelligent Transportation Systems (ITSs) that aims to replace conventional traffic signals with more efficient cooperative systems. Accordingly, it relays on vehicular communication for sharing awareness messages and hence to enable Autonomous Vehicles (AVs) to make effective control decision to optimize their traffic delay. Nonetheless, little focus has been placed on the insertion of bicycles in the ATM. Moreover, no research, to our knowledge, has studied the benefit of the application of Deep Reinforcement Learning (DRL) to integrate cyclists in these distributed systems. To fill this gap, we propose first a Markov Decision Process (MDP) modeling for this scenario. Then, we investigate the use of Deep Q-Learning to optimize vehicle time loss. Finally, we discuss deeply the performance of our proposed DRL-based solution compared to similar traditional programming-based systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
I. Bäumler, H. Kotzab, The emergence of intelligent transportation systems from a continental and technological perspective. World Rev. Intermodal. Transp. Res. 9, 199–216 (2020). https://doi.org/10.1504/WRITR.2020.108220
S. El Hamdani, N. Benamar, A comprehensive study of intelligent transportation system architectures for road congestion avoidance. Lect. Notes Comput. Sci. (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 10542 LNCS:95–106. (2017), doi:https://doi.org/10.1007/978-3-319-68179-5_9
A. Boukerche, J. Wang, Machine learning-based traffic prediction models for intelligent transportation systems. Comput. Netw. 181, 107530 (2020). https://doi.org/10.1016/j.comnet.2020.107530
Y. Lian, G. Zhang, J. Lee, H. Huang, Review on big data applications in safety research of intelligent transportation systems and connected/automated vehicles. Accid. Anal. Prev. 146, 105711 (2020). https://doi.org/10.1016/j.aap.2020.105711
S. Kaffash, A.T. Nguyen, J. Zhu, Big data algorithms and applications in intelligent transportation system: a review and bibliometric analysis. Int. J. Prod. Econ. 231, 107868 (2021). https://doi.org/10.1016/j.ijpe.2020.107868
F. Arena, G. Pau, A. Severino, A review on IEEE 802.11p for intelligent transportation systems. J. Sens Actuator Networks 9, 1–11 (2020). https://doi.org/10.3390/jsan9020022
D. Sirohi, N. Kumar, P.S. Rana, Convolutional neural networks for 5G-enabled intelligent transportation system: a systematic review. Comput. Commun. 153, 459–498 (2020). https://doi.org/10.1016/j.comcom.2020.01.058
M. Ouaissa, M. Houmer, M. Ouaissa, An enhanced authentication protocol based group for vehicular communications over 5G networks, in 3rd Int Conf Adv Commun Technol Networking, CommNet, (2020). doi:https://doi.org/10.1109/CommNet49926.2020.9199641
I. Laña, J.J. Sanchez-Medina, E.I. Vlahogianni, S.J. Del, From data to actions in intelligent transportation systems: a prescription of functional requirements for model actionability. arXiv (2020)
M. Ouaissa, M. Ouaissa, M. Houmer, Improved MAC design-based dynamic duty cycle for vehicular communications over M2M system, in: Springer (ed) Data Analytics and Management. (Singapore, 2021), pp 111–120
S. Hamdani El, Benamar N, Autonomous traffic management: open issues and new directions: International conference on selected topics in mobile and wireless networking (MoWNeT), pp. 1–5 (2018). https://doi.org/10.1109/MoWNet.2018.8428937
S. El Hamdani, N. Benamar, M. Younis, Pedestrian support in intelligent transportation systems: challenges, solutions and open issues. Transp. Res. Part C Emerg. Technol. 121, 102856 (2020). https://doi.org/10.1016/j.trc.2020.102856
S. El Hamdani, N. Benamar, M. Younis, A protocol for pedestrian crossing and increased vehicular flow in smart cities. J. Intell. Transp. Syst. Technol. Planning, Oper. 0, 1–20 (2019). https://doi.org/10.1080/15472450.2019.1683451
S. ElHamdani, N. Benamar, DBDA: distant bicycle detection and avoidance protocol based on V2V communication for autonomous vehicle-bicycle road share, in 2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems, WITS 2019. IEEE, (2019), pp. 1–6
J. Kovaceva, A. Bálint, R. Schindler, A. Schneider, Safety benefit assessment of autonomous emergency braking and steering systems for the protection of cyclists and pedestrians based on a combination of computer simulation and real-world test results. Accid. Anal. Prev. 136, 105352 (2020). https://doi.org/10.1016/j.aap.2019.105352
B.Q. Huang, G.Y. Cao, M. Guo, Reinforcement learning neural network to the problem of autonomous mobile robot obstacle avoidance. Int. Conf. Mach. Learn. Cybern. ICMLC 2005, 85–89 (2005). https://doi.org/10.1109/icmlc.2005.1526924
R. Emuna, A. Borowsky, A. Biess, Deep reinforcement learning for human-like driving policies in collision avoidance tasks of self-driving cars. arXiv (2020)
Z. Sui, Z. Pu, J. Yi, T. Xiong, Formation control with collision avoidance through deep reinforcement learning using model-guided demonstration. Proc. Int. Jt. Conf. Neural Networks, (2019) pp. 1–15. doi:https://doi.org/10.1109/IJCNN.2019.8851906
T. Fan, P. Long, W. Liu, J. Pan, Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. Int. J. Robot. Res. 39, 856–892 (2020). https://doi.org/10.1177/0278364920916531
P. Wang, C.Y. Chan, Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge. arXiv (2017)
J. Li, L. Yao, X. Xu, et al., Deep reinforcement learning for pedestrian collision avoidance and human-machine cooperative driving. Inf. Sci. (Ny) 532, 110–124 (2020). https://doi.org/10.1016/j.ins.2020.03.105
M. Everett, Y.F. Chen, J.P. How, Collision avoidance in pedestrian-rich environments with deep reinforcement learning. IEEE Access 9, 10357–10377 (2021). https://doi.org/10.1109/ACCESS.2021.3050338
S.S. Mousavi, M. Schukat, E. Howley, Deep reinforcement learning: an overview. Lect. Notes Networks Syst. 16, 426–440 (2018). https://doi.org/10.1007/978-3-319-56991-8_32
E.N. Barron, H. Ishii, The Bellman equation for minimizing the maximum cost. Nonlinear Anal. 13, 1067–1090 (1989). https://doi.org/10.1016/0362-546X(89)90096-5
T. Hester, M. Vecerik, O. Pietquin, et al., Deep q-learning from demonstrations. arXiv, 3223–3230 (2017)
K.G. Sheela, S.N. Deepa, Selection of number of hidden neurons in neural networks in renewable energy systems. J. Sci. Ind. Res. (India) 73, 686–688 (2014)
A. Araujo, B. Negrevergne, Y. Chevaleyre, J. Atif, On the Expressive Power of Deep Fully Circulant Neural Networks. arXiv (2017)
S. Chang, T. Cohen, B. Ostdiek, What is the machine learning? Phys. Rev. D 97, 56009 (2018). https://doi.org/10.1103/PhysRevD.97.056009
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Hamdani, S.E., Loudari, S., Ouaissa, M., Ouaissa, M., Benamar, N. (2022). Deep Reinforcement Learning Modeling of a V2V Communication-Based Bike Avoidance Protocol for Increased Vehicular Flow. In: Ouaissa, M., Boulouard, Z., Ouaissa, M., Guermah, B. (eds) Computational Intelligence in Recent Communication Networks . EAI/Springer Innovations in Communication and Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-77185-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-77185-0_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77184-3
Online ISBN: 978-3-030-77185-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)