Abstract
Combinatorial search methods often exhibit a large variability in performance. We study the cost profiles of combinatorial search procedures. Our study reveals some intriguing properties of such cost profiles. The distributions are often characterized by very long tails or “heavy tails”. We will show that these distributions are best characterized by a general class of distributions that have no moments (i.e., an infinite mean, variance, etc.). Such non-standard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We believe this is the first finding of these distributions in a purely computational setting. We also show how random restarts can effectively eliminate heavy-tailed behavior, thereby dramatically improving the overall performance of a search procedure.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Anderson, L. (1985). Completing Partial Latin Squares. Mathematisk Fysiske Meddelelser, 41, 1985, 23–69.
Brelaz, D. (1979). New methods to color the vertices of a graph. Comm. of the ACM (1979) 251–256.
Chambers, John M., Mallows, C.L., and Stuck, B.W. (1976) A method for simulating stable random variables. Journal of the American Statistical Association 71, 340–344.
Cheeseman, Peter and Kanefsky, Bob and Taylor, William M. (1991). Where the Really Hard Problems Are. Proceedings IJCAI-91, 1991, 163–169.
Colbourn, C. (1983). Embedding Partial Steiner Triple Systems is NP-Complete. J. Combin. Theory (A) 35 (1983), 100–105.
Dechter, R. (1991) Constraint networks. Encyclopedia of Artificial Intelligence John Wiley, New York (1991) 276–285.
de Lima, Pedro J.F. (1997). On the robustness of nonlinearity tests to moment condition failure. Journal of Econometrics 76, 251–280.
Denes, J. and Keedwell, A. (1974) Latin Squares and their Applications. Akademiai Kiado, Budapest, and English Universities Press, London, 1974.
Frost, Daniel, Rish, Irina, and Vila, Lluís (1997) Summarizing CSP hardness with continuous probability distributions. Proc. AAAI-97.
Fujita, M., Slaney, J., and Bennett, F. (1993). Automatic Generation of Some Results in Finite Algebra Proc. IJCAI, 1993.
Freuder, E. and Mackworth, A. (Eds.). Constraint-basedreasoning. MIT Press, Cambridge, MA, USA, 1994.
Gent, I. P. and Walsh, T.. (1993). Easy Problems are Sometimes Hard, the DIMACS Challenge on Satisfiability Testing. Piscataway, NJ, Oct. 1993. Full version AIJ (Hogg et al. 1996).
Gent, I. and Walsh, T. (1996) The Satisfiability Constraint Gap. Artificial Intelligence, 81, 1996.
Gomes, C.P. and Selman, B. (1997a) Problem Structure in the Presence of Perturbations Proc. AAAI-97, Providence, RI, 1997.
Gomes, C.P. and Selman, B. (1997b) Algorithm Portfolio Design: Theory vs. Practice, Proc. UAI-97, Providence, RI, 1997.
Hall, Peter (1982) On some simple estimates of an exponent of regular variation. Journal of the Royal Statistical Society, B 44, 37–42.
Huberman, B.A., Lukose, R.M., and Hogg, T. (1997). An economics approach to hard computational problems. Science, 265, 51–54.
Hogg, T., Huberman, B.A., and Williams, C.P. (Eds.) (1996). Phase Transitions and Complexity. Artificial Intelligence, 81 (Spec. Issue; 1996)
Kirkpatrick, S. and Selman, B. (1994) Critical Behavior in the Satisfiability of Random Boolean Expressions. Science, 264 (May 1994) 1297–1301.
Kwan, Alvin C. M. (1995) Validity of normality assumption in CSP research, PRICAI'96: Topics in Artificial Intelligence. Proceedings of the 4th Pacific Rim International Conference on Artificial Intelligence, 459–465.
Lam, C., Thiel, L., and Swiercz, S. (1989) The Non-existence of Finite Projective Planes of Order 10. Can. J. Math., Vol. XLI, 6, 1989, 1117–1123.
Mandelbrot, Benoit B. (1960) The Pareto-Lévy law and the distribution of income. International Economic Review 1, 79–106.
Mandelbrot, Benoit B. (1963) The variation of certain speculative prices. Journal of Business 36, 394–419.
Mandelbrot, B. (1983) The fractal geometry of nature. Freeman: New York. 1983.
Mitchell, D., Selman, B., and Levesque, H.J. (1989) Hard and easy distributions of SAT problems. Proc. AAAI-92, San Jose, CA (1992) 459–465.
Puget, J.-F. (1994) A C++ Implementation of CLP. Technical Report 94-01 ILOG S.A., Gentilly, France, (1994).
Russell, S and Norvig P. (1995) Artificial Intelligence a Modern Approach. Prentice Hall, Englewood Cliffs, NJ. (1995).
Samorodnitsky, Gennady and Taqqu, Murad S. (1994) Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance, Chapman and Hall, New York.
Selman, B. and Kirkpatrick, S. (1996) Finite-Size Scaling of the Computational Cost of Systematic Search. Artificial Intelligence, Vol. 81, 1996, 273–295.
Smith, B. and Dyer, M. Locating the Phase Transition in Binary Constraint Satisfaction Problems. Artificial Intelligence, 81, 1996.
Smith, B. and Grant S.A., Sparse Constraint Graphs and Exceptionally Hard Problems. IJCAI-95, 646–651, 1995. Full version in AIJ (Hogg et al. 1996).
van Hentenryck, P., Deville, Y., and Teng Choh-Man (1992) A generic arc consistency algorithm and its specializations. Artificial Intelligence, 57, 1992.
Williams, C.P. and Hogg, T. (1992) Using deep structure to locate hard problems. Proc. AAAI-92, San Jose, CA, July 1992, 472–277.
Zhang, W. and Korf, R. A Study of Complexity Transitions on the Asymmetric Travelling Salesman Problem. Artificial Intelligence, 81, 1996.
Zolotarev, V.M. (1986) One-dimensional Stable Distributions. Vol. 65 of “Translations of mathematical monographs”, American Mathematical Society. Translation from the original 1983 Russian Ed.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gomes, C.P., Selman, B., Crato, N. (1997). Heavy-tailed distributions in combinatorial search. In: Smolka, G. (eds) Principles and Practice of Constraint Programming-CP97. CP 1997. Lecture Notes in Computer Science, vol 1330. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0017434
Download citation
DOI: https://doi.org/10.1007/BFb0017434
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63753-0
Online ISBN: 978-3-540-69642-1
eBook Packages: Springer Book Archive