Skip to main content
SpringerLink
Log in

Reinforcement Learning for Autonomous Driving Scenarios in Indian Roads

  • Conference paper
  • First Online:
Inventive Communication and Computational Technologies

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 383))

  • 478 Accesses

Abstract

The decision-making process for autonomous vehicles comes with numerous challenges that are not easily solved. With the ever-changing traffic situation of the world and the increasing need for autonomous driving technology, there are constant innovations to deal with the increasing number of problems in the complex environments that autonomous driving agents find themselves in. Developing countries like India face even more numerous challenges with existing autonomous driving solutions not being directly transferable. However, with the maturation and advancement of deep learning technology over the years, more and more novel methods in the field of deep reinforcement learning are being proposed to tackle both new challenges and existing challenges. In this study, we explore the contemporary reinforcement learning techniques for autonomous driving tasks and analyze their applicability for the unstructured road environment and also look at some of the less-common scenarios that occur frequently in the Indian context.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kiran BR, Sobh I, Talpaert V, Mannion P, Sallab AAA, Yogamani S, Pérez P (2021) Deep reinforcement learning for autonomous driving: a survey. IEEE Trans Intell Transp Syst 1–18

    Google Scholar 

  2. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. A Bradford Book, Cambridge, MA, USA

    MATH  Google Scholar 

  3. Clifton J, Laber E (2020) Q-learning: theory and applications. Ann Rev Stat Appl 7:279–301

    Article  MathSciNet  Google Scholar 

  4. Paden B, Cáp M, Yong SZ, Yershov DS, Frazzoli E (2016) A survey of motion planning and control techniques for self-driving urban vehicles. CoRR abs/1604.07446. http://arxiv.org/abs/1604.07446

  5. Dijkstra EW (1959) A note on two problems in connexion with graphs. Numer Math 1(1):269–271. https://doi.org/10.1007/BF01386390

  6. Nilsson NJ (1969) A mobile automaton: an application of artificial intelligence techniques. In: IJCAI

    Google Scholar 

  7. Ye F, Zhang S, Wang P, Chan CY (2021) A survey of deep reinforcement learning algorithms for motion planning and control of autonomous vehicles. arXiv preprint arXiv:2105.14218

  8. Sallab AE, Abdou M, Perot E, Yogamani S (2017) Deep reinforcement learning framework for autonomous driving. Electronic Imaging 2017(19):70–76

    Article  Google Scholar 

  9. Okuyama T, Gonsalves T, Upadhay J (2018) Autonomous driving system based on deep q learnig. In: 2018 International Conference on Intelligent Autonomous Systems (ICoIAS), pp 201–205

    Google Scholar 

  10. Isele D, Nakhaei A, Fujimura K (2019) Safe reinforcement learning on autonomous vehicles. CoRR abs/1910.00399. http://arxiv.org/abs/1910.00399

  11. Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. In: Balcan MF, Weinberger KQ (eds) Proceedings of The 33rd international conference on machine learning. Proceedings of machine learning research, vol 48, pp 1928–1937. PMLR, New York, USA, 20–22 Jun 2016. https://proceedings.mlr.press/v48/mniha16.html

  12. Espié E, Guionneau C, Wymann B, Dimitrakakis C, Coulom R, Sumner A (2005) Torcs, the open racing car simulator

    Google Scholar 

  13. Chen J, Yuan B, Tomizuka M (2019) Model-free deep reinforcement learning for urban autonomous driving. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp 2765–2771

    Google Scholar 

  14. Dosovitskiy A, Ros G, Codevilla F, Lopez A, Koltun V (2017) CARLA: An open urban driving simulator. In: Proceedings of the 1st annual conference on robot learning, pp 1–16

    Google Scholar 

  15. Vlachogiannis DM, Vlahogianni EI, Golias J (2020) A reinforcement learning model for personalized driving policies identification. Int J Transp Sci Technol 9(4):299–308. https://www.sciencedirect.com/science/article/pii/S2046043020300198

  16. Min K, Kim H, Huh K (2019) Deep distributional reinforcement learning based high-level driving policy determination. IEEE Trans Intell Veh 4(3):416–424

    Article  Google Scholar 

  17. Zhu M, Wang Y, Pu Z, Hu J, Wang X, Ke R (2020) Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transp Res Part C: Emerging Technol 117:102662. https://www.sciencedirect.com/science/article/pii/S0968090X20305775

  18. Nishi T, Doshi P, Prokhorov D (2019) Merging in congested freeway traffic using multipolicy decision making and passive actor-critic learning. IEEE Trans Intell Veh 4(2):287–297

    Article  Google Scholar 

  19. Li C, Czarnecki K (2018) Urban driving with multi-objective deep reinforcement learning. CoRR abs/1811.08586. http://arxiv.org/abs/1811.08586

  20. Xiong X, Wang J, Zhang F, Li K (2016) Combining deep reinforcement learning and safety based control for autonomous driving. ArXiv abs/1612.00147

    Google Scholar 

  21. Indian roads dataset, open government data (ogd) platform India, https://data.gov.in/dataset-group-name/roads. Accessed 02 Feb 2022

  22. Schoettle B, Sivak M (2014) Public opinion about self-driving vehicles in China, India, Japan, The US, The UK, and Australia. University of Michigan, Ann Arbor, Transportation Research Institute, Tech rep

    Google Scholar 

  23. Zhang P, Xiong L, Yu Z, Fang P, Yan S, Yao J, Zhou Y (2019) Reinforcement learning-based end-to-end parking for automatic parking system. Sensors 19(18). https://www.mdpi.com/1424-8220/19/18/3996

  24. Thunyapoo B, Ratchadakorntham C, Siricharoen P, Susutti W (2020) Self-parking car simulation using reinforcement learning approach for moderate complexity parking scenario. In: 2020 17th international conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp 576–579

    Google Scholar 

  25. Zhang J, Chen H, Song S, Hu F (2020) Reinforcement learning-based motion planning for automatic parking system. IEEE Access 8:154485–154501

    Article  Google Scholar 

  26. Du Z, Miao Q, Zong C (2020) Trajectory planning for automated parking systems using deep reinforcement learning. Int J Autom Technol 21:881–887

    Google Scholar 

  27. Liao J, Liu T, Tang X, Mu X, Huang B, Cao D (2020) Decision-making strategy on highway for autonomous vehicles using deep reinforcement learning. IEEE Access 8:177804–177814

    Article  Google Scholar 

  28. Wang G, Hu J, Li Z, Li L (2020) Harmonious lane changing via deep reinforcement learning. IEEE Trans Intell Transp Syst 1–9

    Google Scholar 

  29. Krasowski H, Wang X, Althoff M (2020) Safe reinforcement learning for autonomous lane changing using set-based prediction. In: 2020 IEEE 23rd international conference on Intelligent Transportation Systems (ITSC), pp 1–7

    Google Scholar 

  30. Triest S, Villaflor A, Dolan JM (2020) Learning highway ramp merging via reinforcement learning with temporally-extended actions. In: 2020 IEEE Intelligent Vehicles symposium (IV), pp 1595–1600

    Google Scholar 

  31. Nishitani I, Yang H, Guo R, Keshavamurthy S, Oguchi K (2020) Deep merging: vehicle merging controller based on deep reinforcement learning with embedding network. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 216–221

    Google Scholar 

  32. Kim M, Lee S, Lim J, Choi J, Kang SG (2020) Unexpected collision avoidance driving strategy using deep reinforcement learning. IEEE Access 8:17243–17252

    Article  Google Scholar 

  33. Kontes GD, Scherer DD, Nisslbeck T, Fischer J, Mutschler C (2020) High-speed collision avoidance using deep reinforcement learning and domain randomization for autonomous vehicles. In: 2020 IEEE 23rd international conference on Intelligent Transportation Systems (ITSC), pp 1–8

    Google Scholar 

  34. Arvind CS, Senthilnath J (2019) Autonomous rl: autonomous vehicle obstacle avoidance in a dynamic environment using mlp-sarsa reinforcement learning. In: 2019 IEEE 5th International Conference on Mechatronics System and Robots (ICMSR), pp 120–124

    Google Scholar 

  35. Fuchs F, Song Y, Kaufmann E, Scaramuzza D, Durr P (2021) Super-human performance in gran turismo sport using deep reinforcement learning. IEEE Robot Autom Lett 6(3):4257–4264. https://doi.org/10.1109/LRA.2021.3064284

  36. Niu J, Hu Y, Jin B, Han Y, Li X (2020) Two-stage safe reinforcement learning for high-speed autonomous racing. In: 2020 IEEE international conference on Systems, Man, and Cybernetics (SMC), pp 3934–3941

    Google Scholar 

  37. Güçkıran K, Bolat B (2019) Autonomous car racing in simulation environment using deep reinforcement learning. In: 2019 innovations in intelligent systems and applications conference (ASYU), pp 1–6

    Google Scholar 

  38. Jaritz M, de Charette R, Toromanoff M, Perot E, Nashashibi F (2018) End-to-end race driving with deep reinforcement learning. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp 2070–2075

    Google Scholar 

  39. Balaji B, Mallya S, Genc S, Gupta S, Dirac L, Khare V, Roy G, Sun T, Tao Y, Townsend B, Calleja E, Muralidhara S, Karuppasamy D (2020) Deepracer: autonomous racing platform for experimentation with sim2real reinforcement learning. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 2746–2754

    Google Scholar 

  40. Kaushik M, Prasad V, Krishna KM, Ravindran B (2018) Overtaking maneuvers in simulated highway driving using deep reinforcement learning. In: 2018 IEEE Intelligent Vehicles symposium (IV), pp 1885–1890

    Google Scholar 

  41. Li X, Xu X, Zuo L (2015) Reinforcement learning based overtaking decision-making for highway autonomous driving. In: 2015 sixth International Conference on Intelligent Control and Information Processing (ICICIP), pp 336–342

    Google Scholar 

  42. Mo S, Pei X, Chen Z (2019) Decision-making for oncoming traffic overtaking scenario using double dqn. In: 2019 3rd Conference on Vehicle Control and Intelligence (CVCI), pp 1–4

    Google Scholar 

  43. Zhu M, Wang Y, Pu Z, Hu J, Wang X, Ke R (2020) Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transp Res Part C: Emerging Technol 117:102662

    Google Scholar 

  44. Pozzi A, Bae S, Choi Y, Borrelli F, Raimondo DM, Moura S (2020) Ecological velocity planning through signalized intersections: a deep reinforcement learning approach. In: 2020 59th IEEE Conference on Decision and Control (CDC), pp 245–252

    Google Scholar 

  45. Capasso AP, Bacchiani G, Broggi A (2020) From simulation to real world maneuver execution using deep reinforcement learning. In: 2020 IEEE Intelligent Vehicles symposium (IV), pp 1570–1575

    Google Scholar 

  46. Garcia Cuenca L, Sanz E, Fernández J, Aliane N (2019) Autonomous driving in roundabout maneuvers using reinforcement learning with q-learning. Electronics 8:1536

    Google Scholar 

  47. Hoel CJ, Wolff K, Laine L (2018) Automated speed and lane change decision making using deep reinforcement learning. In: 2018 21st international conference on Intelligent Transportation Systems (ITSC), pp 2148–2155

    Google Scholar 

  48. Deshpande N, Vaufreydaz D, Spalanzani A (2021) Navigation in urban environments amongst pedestrians using multi-objective deep reinforcement learning. In: 2021 IEEE international Intelligent Transportation Systems Conference (ITSC), pp 923–928

    Google Scholar 

  49. Fu Y, Li C, Yu FR, Luan TH, Zhang Y (2020) A decision-making strategy for vehicle autonomous braking in emergency via deep reinforcement learning. IEEE Trans Veh Technol 69(6):5876–5888

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, B.M.S. College of Engineering, Bengaluru, Karnataka, India

    Adithya Narasimhan, Aravindh R. Shankar, Ajay Mittur & N. Kayarvizhy

Authors
  1. Adithya Narasimhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aravindh R. Shankar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ajay Mittur
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. N. Kayarvizhy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adithya Narasimhan .

Editor information

Editors and Affiliations

  1. Department of Electronics and Communication Engineering, Gnanamani College of Technology, Namakkal, Tamil Nadu, India

    G. Ranganathan

  2. Ryerson Communications Lab Department of Electrical and Computer Engineering, Ryerson University, Toronto, ON, Canada

    Xavier Fernando

  3. ISEG, University of Lisbon, Lisboa, Portugal

    Álvaro Rocha

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Narasimhan, A., Shankar, A.R., Mittur, A., Kayarvizhy, N. (2023). Reinforcement Learning for Autonomous Driving Scenarios in Indian Roads. In: Ranganathan, G., Fernando, X., Rocha, Á. (eds) Inventive Communication and Computational Technologies. Lecture Notes in Networks and Systems, vol 383. Springer, Singapore. https://doi.org/10.1007/978-981-19-4960-9_31

Download citation

Publish with us

Policies and ethics

Search

Navigation