Abstract
The deployment of autonomous agents in real applications promises great benefits, but it also risks potentially great harm to humans who interact with these agents. Indeed, in many applications, agent designers pursue adjustable autonomy (AA) to enable agents to harness human skills when faced with the inevitable difficulties in making autonomous decisions. There are two key shortcomings in current AA research. First, current AA techniques focus on individual agent-human interactions, making assumptions that break down in settings with teams of agents. Second, humans who interact with agents want guarantees of safety, possibly beyond the scope of the agent’s initial conception of optimal AA. Our approach to AA integrates Markov Decision Processes (MDPs) that are applicable in team settings, with support for explicit safety constraints on agents’ behaviors. We introduce four types of safety constraints that forbid or require certain agent behaviors. The paper then presents a novel algorithm that enforces obedience of such constraints by modifying standard MDP algorithms for generating optimal policies. We prove that the resulting algorithm is correct and present results from a real-world deployment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Isaac Asimov. Runaround. Astounding Science Fiction, pages 94–103, 1942.
Craig Boutilier, Ray Reiter, Mikhail Soutchanksi, and Sebastian Thrun. Decision-theoretic, high-level agent programming in the situation calculus. In Proceedings of the National Conference on Artificial Intelligence, pages 355–362, 2000.
Gregory A. Dorais, R. Peter Bonasso, David Kortenkamp, Barney Pell, and Debra Schreckenghost. Adjustable autonomy for human-centered autonomous systems on mars. In Proceedings of the International Conference of the Mars Society, 1998.
George Ferguson, James Allen, and Brad Miller. TRAINS-95: towards a mixed initiative planning assistant. In Proceedings of the Conference on Artificial Intelligence Planning Systems, pages 70–77, May 1996.
Diana F. Gordon. Asimovian adaptive agents. Journal of Artificial Intelligence Research, 13:95–153, 2000.
Victor Lesser, Michael Atighetchi, Brett Benyo, Bryan Horling, Anita Raja, Regis Vincent, Thomas Wagner, P. Xuan, and S. Zhang. The UMASS intelligent home project. In Proceedings of the Conference on Autonomous Agents, pages 291–298, Seattle, USA, 1999.
Ronald Parr and Stuart Russell. Reinforcement learning with hierarchies of machines. In Proceedings of Advances in Neural Information Processing Systems, pages 53–79, 1997.
Martin L. Puterman. Markov Decision Processes. John Wiley & Sons, 1994.
David V. Pynadath, Milind Tambe, Yigal Arens, Hans Chalupsky, Yolanda Gil, Craig Knoblock, Haeyoung Lee, Kristina Lerman, Jean Oh, Surya Ramachandran, Paul S. Rosenbloom, and Thomas Russ. Electric Elves: Immersing an agent organization in a human organization. In AAAI Fall Symposium on Socially Intelligent Agents: The Human in the Loop, pages 150–154, 2000.
Paul Scerri, David V. Pynadath, and Milind Tambe. Adjustable autonomy in real-world multi-agent environments. In Proceedings of the Conference on Autonomous Agents, pages 300–307, 2001.
Milind Tambe. Towards flexible teamwork. Journal of Artificial Intelligence Research, 7:83–124, 1997.
Daniel S. Weld and Oren Etzioni. The first law of robotics (a call to arms). In Proceedings of the National Conference on Artificial Intelligence, pages 1042–1047, 1994.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pynadath, D.V., Tambe, M. (2002). Revisiting Asimov’s First Law: A Response to the Call to Arms. In: Meyer, JJ.C., Tambe, M. (eds) Intelligent Agents VIII. ATAL 2001. Lecture Notes in Computer Science(), vol 2333. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45448-9_22
Download citation
DOI: https://doi.org/10.1007/3-540-45448-9_22
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43858-8
Online ISBN: 978-3-540-45448-9
eBook Packages: Springer Book Archive