Abstract
Many real-world decision problems require making trade-offs between multiple objectives. However, in some cases, the relative importance of the objectives is not known when the problem is solved, precluding the use of single-objective methods. Instead, multi-objective methods, which compute the set of all potentially useful solutions, are required. This paper proposes new multi-objective algorithms for cooperative multi-agent settings. Following previous approaches, we exploit loose couplings, as expressed in graphical models, to coordinate efficiently. Existing methods, however, calculate only the Pareto coverage set (PCS), which we argue is inappropriate for stochastic strategies and unnecessarily large when the objectives are weighted in a linear fashion. In these cases, the typically much smaller convex coverage set (CCS) should be computed instead. A key insight of this paper is that, while computing the CCS is more expensive in unstructured problems, in many loosely coupled settings it is in fact cheaper to compute because the local solutions are more compact. We propose convex multi-objective variable elimination, which exploits this insight. We analyze its correctness and complexity and demonstrate empirically that it scales much better in the number of agents and objectives than alternatives that compute the PCS.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Barrett, L., Narayanan, S.: Learning all optimal policies with multiple criteria. In: ICML, pp. 41–47. ACM, New York (2008)
Becker, R., Zilberstein, S., Lesser, V., Goldman, C.V.: Transition-Independent Decentralized Markov Decision Processes. In: AAMAS (2003)
Brázdil, T., Brozek, V., Chatterjee, K., Forejt, V., Kucera, A.: Two views on multiple mean-payoff objectives in Markov decision processes. CoRR, abs/1104.3489 (2011)
Cassandra, A.R., Littman, M.L., Zhang, N.L.: Incremental pruning: A simple, fast, exact method for partially observable markov decision processes. In: UAI, pp. 54–61 (1997)
Delle Fave, F.M., Stranders, R., Rogers, A., Jennings, N.R.: Bounded decentralised coordination over multiple objectives. In: AAMAS, pp. 371–378 (2011)
Dubus, J.-P., Gonzales, C., Perny, P.: Choquet optimization using gai networks for multiagent/multicriteria decision-making. In: Rossi, F., Tsoukias, A. (eds.) ADT 2009. LNCS, vol. 5783, pp. 377–389. Springer, Heidelberg (2009)
Feng, Z., Zilberstein, S.: Region-based incremental pruning for POMDPs. CoRR, abs/1207.4116 (2012)
Guestrin, C.E., Koller, D., Parr, R.: Multiagent planning with factored MDPs. In: NIPS (2002)
Kok, J.R., Vlassis, N.: Collaborative multiagent reinforcement learning by payoff propagation. J. Mach. Learn. Res. 7, 1789–1828 (2006)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)
Lizotte, D.J., Bowling, M., Murphy, S.A.: Efficient reinforcement learning with multiple reward functions for randomized clinical trial analysis. In: ICML, pp. 695–702 (2010)
Marinescu, R., Razak, A., Wilson, N.: Multi-objective influence diagrams. In: UAI (2012)
Roijers, D.M., Whiteson, S., Oliehoek, F.A.: Multi-objective variable elimination for collaborative graphical games. In: AAMAS (2013) (Extended Abstract)
Rollón, E.: Multi-Objective Optimization for Graphical Models. PhD thesis, Universitat Politècnica de Catalunya (2008)
Rollón, E., Larrosa, J.: Bucket elimination for multiobjective optimization problems. Journal of Heuristics 12, 307–328 (2006)
Tesauro, G., Das, R., Chan, H., Kephart, J.O., Lefurgy, C., Levine, D.W., Rawson, F.: Managing power consumption and performance of computing systems using reinforcement learning. In: NIPS (2007)
Vamplew, P., Dazeley, R., Barker, E., Kelarev, A.: Constructing stochastic mixture policies for episodic multiobjective reinforcement learning tasks. In: Nicholson, A., Li, X. (eds.) AI 2009. LNCS, vol. 5866, pp. 340–349. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Roijers, D.M., Whiteson, S., Oliehoek, F.A. (2013). Computing Convex Coverage Sets for Multi-objective Coordination Graphs. In: Perny, P., Pirlot, M., Tsoukiàs, A. (eds) Algorithmic Decision Theory. ADT 2013. Lecture Notes in Computer Science(), vol 8176. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41575-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-642-41575-3_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41574-6
Online ISBN: 978-3-642-41575-3
eBook Packages: Computer ScienceComputer Science (R0)