Abstract
In order to accomplish it successfully, the top-level goal of a multi-robot team should be decomposed into a sequence of sub-goals and proper sequences of actions for achieving these subgoals should be selected and refined through execution. Selecting the proper actions at any given time requires the ability to evaluate the current state of the environment, which can be achieved by using metrics that give quantitative information about the environment. Defining appropriate metrics is already a challenging problem; however, it is even harder to assess the performance of individual metrics. This work proposes a layered evaluation scheme for robot soccer where the environment is represented in different time resolutions at each layer. A set of metrics defined on these layers together with a novel metric validation method for assessing the performance of the defined metrics are proposed.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
VerDuin, W.H., Kothamasu, R., Huang, S.H.: Analysis of performance evaluation metrics to combat the model selection problem. In: PERMISA Workshop (2003)
Yavnai, A.: An information-based approach for system autonomy metrics part i: Metrics definition. In: PERMISA Workshop (2003)
Horst, J.A.: Exercising a native intelligence metric on an autonomous on-road driving system. In: PERMISA Workshop (2003)
Olsen, D.R., Goodrich, M.A.: Metrics for evaluating human-robot interactions. In: PERMISA Workshop (2003)
Balch, T.: Social entropy: a new metric for learning multi-robot teams. In: 10th International FLAIRS Conference (FLAIRS 1997) (1997)
Rue, H., Salvesen, Ø.: Predicting and retrospective analysis of soccer matches in a league
Yanco, H.A.: Designing metrics for comparing the performance of robotic systems in robot competitions. In: Workshop on Measuring Performance and Intelligence of Intelligent Systems (PERMIS) (2001)
Kok, J., de Boer, R., Vlassis, N.: Towards an optimal scoring policy for simulated soccer agents. Technical report (2001)
Dylla, F., Ferrein, A., Lakemeyer, G., Murray, J., Obst, O., Röfer, T., Stolzenburg, F., Visser, U., Wagner, T.: Towards a league-independent qualitative soccer theory for robocup. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds.) RoboCup 2004. LNCS, vol. 3276, pp. 611–618. Springer, Heidelberg (2005)
Veloso, M., Lenser, S., Vail, D., Roth, M., Stroupe, A., Chernova, S.: Cmpack 2002: Cmu’s legged robot soccer team. Technical report (2002)
Röfer, T., et al.: Germanteam 2006. Technical report, The GermanTeam (2007)
Quinlan, M.J., Henderson, N., Middleton, R.H.: The 2006 nubots team report. Technical report, Newcastle Robotics Laboratory (2007)
Balch, T.: Teambots (2000), http://www.teambots.org
Cohen, P.R.: Empirical Methods for Artificial Intelligence. MIT Press, Massachusetts (1995)
Hodrick, R.J., Prescott, E.C.: Postwar u.s. business cycles: An empirical investigation. Journal of Money, Credit and Banking 29 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Meriçli, Ç., Akın, H.L. (2009). A Layered Metric Definition and Evaluation Framework for Multirobot Systems. In: Iocchi, L., Matsubara, H., Weitzenfeld, A., Zhou, C. (eds) RoboCup 2008: Robot Soccer World Cup XII. RoboCup 2008. Lecture Notes in Computer Science(), vol 5399. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02921-9_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-02921-9_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02920-2
Online ISBN: 978-3-642-02921-9
eBook Packages: Computer ScienceComputer Science (R0)