Abstract
Bridge bidding is considered to be one of the most difficult problems for game-playing programs. It involves four agents rather than two, including a cooperative agent. In addition, the partial observability of the game makes it impossible to predict the outcome of each action. In this paper we present a new decision-making algorithm that is capable of overcoming these problems. The algorithm allows models to be used for both opponent agents and partners, while utilizing a novel model-based Monte Carlo sampling method to overcome the problem of hidden information. The paper also presents a learning framework that uses the above decision-making algorithm for co-training of partners. The agents refine their selection strategies during training and continuously exchange their refined strategies. The refinement is based on inductive learning applied to examples accumulated for classes of states with conflicting actions. The algorithm was empirically evaluated on a set of bridge deals. The pair of agents that co-trained significantly improved their bidding performance to a level surpassing that of the current state-of-the-art bidding algorithm.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Ando, T., Kobayashi, N., & Uehara. T. (2003). Cooperation and competition of agents in the auction of computer bridge. Electronics and Communications in Japan, 86(12).
Belladonna, G., & Garozzo. B. (1975). Precision & super precision bidding. Cassell, London.
Billings, D., Burch, N., Davidson, A., Holte, R., Schaeffer, J., Schauenberg, T., & Szafron, D. (2003). Approximating game-theoretic optimal strategies for fullscale poker. In: Proceedings of the Eighteens International Joint Conference on Artificial Intelligence (pp. 661–668).
Billings, D., Davidson, A., Schaeffer, J., & Szafron, D. (2002). The challenge of poker. Artificial Intelligence 134:1-2, 201–240.
Billings, D., Papp, D., Schaeffer, J., & Szafron, D. (1998). Opponent modeling in poker. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence. (pp. 493–499).
Billings, D., Pena, L., Schaeffer, J., & Szafron, D. (1998). Using probabilistic knowledge and simulation to play poker. In: Proceedings of the Sixteenth National Conference on Artificial Intelligence (pp. 697–703).
Bird, R., (1999). BTM’s first steps into computing. Computer Resurrection: The Bulletin of the Computer Conservation Society, 22, 12–18.
Bruce, J., Bowling, M., Browning, B., & Veloso, M. (2002). Multi-robot team response to a multi-robot opponent team. In: Proceedings of the 2003 IEEE International Conference on Robotics and Automation (pp. 2281–2286).
Carley, G. (1962). A program to play contract bridge. Master’s thesis, Dept. of Electrical Engineering, Massachussets Institute of Technology, Cambridge, Massachussets.
Carmel, D., & Markovitch, S. (1996a). Incorporating opponent models into adversary search. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence (pp. 120–125). Portland, Oregon.
Carmel, D., & Markovitch, S. (1996b). Learning and using opponent models in adversary search. Technical Report CIS9609, Technion.
Carmel, D., & Markovitch, S. (1998). Model-based learning of interaction strategies in multi-agent systems. Journal of Experimental and Theoretical Artificial Intelligence, 10:3, 309–332.
Cohen, L. (2002). Larry Cohen’s bidding challenge. Master Point Print, Toronto.
Davidson, A., Billings, D., Schaeffer, J., & Szafron, D. (2000). Improved opponent modeling in poker. In: Proceedings of the 2000 International Conference on Artificial Intelligence (pp. 1467–1473).
Donkers, H. (2003). Nosce Hostem –- Searching with opponent models. Ph.D. thesis, Universiteit Maastricht.
Donkers, H., Uiterwijk, J., & van, den Herik, H. (2001). Probabilistic opponent-model search. Information Sciences, 135:3-4, 123–149.
Eskes, O. (1998). The par contest. Imp Magazine, 9 (7).
Feigenbaum, E., & Simon, H. (1984). EPAM-like models of recognition and learning. Cognitive Science, 8:43, 305–336.
Finkelstein, L., & Markovitch, S. (1998). Learning to play chess selectively by acquiring move patterns. ICCA Journal, 21:2, 100–119.
Frank, I. (1998). Search and planning under incomplete information: study using bridge card play. Springer.
Gambäck, B., Rayner, M., & Pell, B. (1993). Pragmatic reasoning in bridge. Technical Report CRC-030, SRI International Cambridge Computer Science Reasearch Center.
Gao, X. Iida, H. Uiterwijk, J., & van den Herik. H. J. (1999). A speculative strategy. Lecture Notes in Computer Science, 1558, 74–92.
Ginsberg, M. L. (2001). GIB: Imperfect information in a computationally challenging game. Journal of Artificial Intelligence Research, 14, 303–358.
Goldman, B. (1978). Aces scientific. Max Hardy.
Hyatt, R. M. (1999). Book learning – A methodology to tune an opening book automatically. International Computer Chess Association Journal, 22:1, 3–12.
Iida, H., Uiterwijk, J. W., & van den Herik, H. (2000). Cooperative strategies for pair playing. In: H. van den Herik and H. Iida (eds.), Games in AI Research. Universiteit Maastricht (pp. 189–201).
Iida, H., Uiterwijk, J. W. H., van den Herik, H. J., & Herschberg, I. S., (1993). Potential applications of opponent-model search, Part I: The domain of applicability. ICCA Journal, 16:4, 201–208.
Iida, H., Uiterwijk, J. W. H., van den Herik, H. J., & Herschberg, I. S. (1994). Potential applications of opponent-model search, Part II: risks and strategies. ICCA Journal, 17:1, 10–14.
Jamroga, W. (1999). Modelling artificial intelligence on a case of bridge card play bidding. In: Proceedings of the 8th International Workshop on Intelligent Information Systems (pp. 267–277) Ustron, Poland.
Jamroga, W. (2001). A defense model for games with incomplete information. Lecture Notes in Computer Science, 2174, 260–274.
Jansen, P. J. (1992). Using knowledge about the opponent in game-tree search. Ph.D. thesis, Carnegie Mellon University.
Klinger, R. (2001). Modern losing trick count. Orion.
Levy, D. (1989). The million pound bridge program. In: D. Levy and D. Beal (eds.), Heuristic programming in artificial intelligence—First Computer Olympiad (pp. 93–105).
Lindelöf, E. (1983). COBRA—The computer-designed bidding system. England: Victor Gollancz.
Luckhardt, C. A., & Irani, K. B. (1986). An algorithmic solution of N-person games. In: Proceedings of the Ninth National Conference on Artificial Intelligence (pp. 158–162). Philadelfia, PA.
Lustrek, M., Gams, M., & Bratko, I. (2003). A program for playing tarok. ICGA Journal, 26:3, 190–197.
Macleod, J. (1989). Microbridge—A Computer Developed Approach to Bidding. In: D. Levy and D. Beal (eds.), Heuristic programming in artificial intelligence—the first computer olympiad (pp. 81–87).
Mahmood, Z. (1994). Bridge my way. Natco Press.
Markovitch, S., & Reger, R. (2005). Learning and exploiting relative weaknesses of opponent agents. Autonomous Agents and Multi-agent Systems, 10:2, 103–130.
Markovitch, S., & Sella, Y. (1996). Learning of resource allocation strategies for game playing. Computational Intelligence, 12:1, 88–105.
Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3:3, 211–229.
Schaeffer, J. (2000). The games computers (and People) play. In: Marvin Zelkowitz (ed.), Advances in computer 50, (pp. 189–266). Academic Press.
Scott, P. D., & Markovitch, S. (1991). Representation generation in an exploratory learning system. In: D. Fisher and M. Pazzani (eds.), Concept formation: Knowledge and experience in unsupervised learning. Morgan Kaufmann.
Scott, P. D., & Markovitch, S. (1993). Experience selection and problem choice in an exploratory learning system. Machine Learning, 12, 49–67.
Sen, S., & Arora, N. (1997). Learning to take risks. In: AAAI-97 Workshop on Multiagent Learning. (pp. 59–64).
Sen, S., & Sekaran, M. (1998). Individual learning of coordination knowledge. Journal of Experimental & Theoretical Artificial Intelligence 10:3, (p. 333–356).
Shi, J., & Littman, M. (2001). Abstraction models for game theoretic poker. Computers and Games, (pp. 333–345).
Smith, S. J. J., Nau, D., & Throop, T. (1996). A planning approach to declarer play in contract bridge. Computational Intelligence, 12:1, 106–130.
Smith, S. J. J., Nau, D., & Throop, T. (1998a). Computer bridge: A big win for AI planning. AI Magazine, 2:19, 93–105.
Smith, S. J. J., Nau, D., & Throop, T. (1998b). Success in spades: Using AI planning techniques to win the world championship of computer bridge. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, (pp. 1079–1086).
Stanier, A. M.. (1975). BRIBIP: A bridge bidding program. In: The Proceedings of the Fourth International Joint Conference on Artificial Intelligence (pp. 374–378).
Sturtevant, N. (2004). Current challenges in multi-player game search. In: Proceedings of Computers and Games. Israel.
Wasserman, A. (1970). Realization of a skillful bridge bidding program. Fall Joint Computer Conference, (pp. 433–444). Houston, Texas.
Author information
Authors and Affiliations
Corresponding author
Additional information
Editors: Michael Bowling, Johannes Fürnkranz, Thore Graepel, Ron Musick
Rights and permissions
About this article
Cite this article
Amit, A., Markovitch, S. Learning to bid in bridge. Mach Learn 63, 287–327 (2006). https://doi.org/10.1007/s10994-006-6225-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-006-6225-2