Abstract
‘The problem with simulations is that they are doomed to succeed.’ So runs a common criticism of simulations—that they can be used to ‘prove’ anything and are thus of little or no scientific value. While this particular objection represents a minority view, especially among those who work with simulations in a scientific context, it raises a difficult question: what standards should we use to differentiate a simulation that fails from one that succeeds? In this paper we build on a structural analysis of simulation developed in previous work to provide an evaluative account of the variety of ways in which simulations do fail. We expand the structural analysis in terms of the relationship between a simulation and its real-world target emphasizing the important role of aspects intended to correspond and also those specifically intended not to correspond to reality. The result is an outline both of the ways in which simulations can fail and the scientific importance of those various forms of failure.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Axelrod R., Hamilton W. (1981) The evolution of cooperation. Science 211: 1390–1396
Axtell, R. L., Epstein, J. M., Dean, J. S., Gumerman, G. J., Swedland, A. C., Harburger, J., Chakravarty, S., Hammond, R., Parker, J., & Parker, M. (2002). Population growth and collapse in a multiagent model of the Kayenta Anasazi in Long House Valley. In B. J. L.Berry, L. D. Kiel, & E. Eliott (Eds.), Adaptive agents, intelligence, and emergent human organization: Capturing complexity through agent-based modeling Proceedings of the National Academy of Sciences of the USA, (Vol. 99, Suppl. 3, pp. 7275–7279). Washington, DC: National Academy of Sciences.
Barberousse A., Franceschelli S., Imbert C. (2009) Computer simulations as expriments. Synthese 169: 557–574
Borges, J. L. (1998). Collected fictions (A. Hurley, Trans.). New York, NY: Penguin Books.
Bottke W., Vokrouhlický D., Nesvorný D. (2007) An asteroid breakup 160 myr ago as the probable source of the K/T impactor. Nature 449: 48–53
Bruch, E. E., & Mare, R. D. (2001). Spatial inequality, neighborhood mobility, and residential segregation. Los Angeles, CA: California Center for Population Research On-Line Working Paper Series.
Bruch E. E., Mare R. D. (2006) Neighborhood choice and neighborhood change. American Journal of Sociology 112: 667–709
Canup R. M. (2004) Simulations of a late lunar forming impact. Icarus 168: 433–456
Cartwright N. (1983) How the laws of physics lie. Oxford University Press, Oxford
Cartwright, N. (1999). Aristotelian natures and modern experimental method. In The dappled world. Cambridge: Cambridge University Press.
Chattoe E., Saam N. J., Möhring M. (2000) Sensitivity analysis in the social sciences: Problems and prospects. In: Suleiman R., Troitsch K. G., Gilbert N. (eds) Tools and techniques for social science simulation. Physica-Verlag, pp 243–273 Heidelberg
Committee on Modeling Community Containment for Pandemic Influenza. (2006). Modeling community containment for pandemic influenza: A letter report. Institute of Medicine of the National Academies. http://www.nap.edu/catalog/11800.html
Cummings D. A. T., Chakravarty S., Singha R. M., Burke D. S., Epstein J. M. (2004) Toward a containment strategy for smallpox bioterror: An individual-based computational approach. Brookings Institute Press, Washington, DC
Da Costa N., French S. (2003) Science and partial truth: A unitary approach to models and scientific reasoning. Oxford University Press, Oxford
Dean J. S., Gumerman G. J., Epstein J., Axtell R. L., Swedland A. C., Parker M. T., McCarrol S. (1999) Understanding Anasazi culture change through agent based modeling. In: Kohler T. A., Gumerman G. J. (eds) Dynamics in human and primate societies: Agent based modeling of social and spatial processes. Oxford University Press, New York, NY, pp 179–206
Eason R., Rosenberger R., Kokalis T., Selinger E., Grim P. (2007) What kind of science is simulation?. Journal of Experimental and Theoretical Artificial Intelligence 19(1): 19–28
Elgin C. (2009) Exemplification, idealization, and scientific understanding. In: Suárez M. (eds) Fictions in science: Philosophical essays on modeling and idealization. Routledge, New York, NY
Epstein J. M. (2002) Modeling civil violence: An agent-based computational approach. Proceedings of the National Academy of Sciences, USA 99: 7243–7250
Ferguson N. M., Cummings D. A. T., Fraser C., Cajka J. C., Cooley P. C., Burke D. S. (2006) Strategies for mitigating an influenza pandemic. Nature 442: 448–451
Frigg, R., & Hartmann, S. (2006). Models in science. Stanford encyclopedia of philosophy. http://plato.stanford.edu/entries/models-science.
Giere R. (2004) How models are used to represent reality. Philosophy of Science 71(Supplement): S742–752
Glymour C., Scheines R., Spirtes P., Kelly K. (1987) Discovering causal structure: Artificial intelligence, philosophy of science, and statistical modeling. Academic Press, San Diego, CA
Granovetter M. (1978) Threshold models of collective behavior. American Sociological Review 83: 1420–1442
Grim P. (1995) Greater generosity in the spatialized Prisoner’s Dilemma. Journal of Theoretical Biology 173: 353–359
Grim P. (1996) Spatialization and greater generosity in the stochastic Prisoner’s Dilemma. Biosystems 37: 3–17
Grim P., Mar G., St. Denis P. (1998) The philosophical computer: Exploratory essays in philosophical computer modeling. MIT Press, Cambridge, MA
Grim P., Au R., Louie N., Rosenberger R., Braynen W., Selinger E., Eason R. E (2006) Game-theoretic robustness in cooperation and prejudice reduction: A graphic measure. In: Rocha L. M., Yaeger L. S., Bedau M. A., Floreano D., Goldstone R. L., Vespignani A. (eds) Artificial life X. MA: MIT Press, Cambridge, pp 445–451
Grim P., Au R., Louie N., Rosenberger R., Braynen W., Selinger E., Eason R. E. (2008) A graphic measure for game theoretic robustness. Synthese 163(2): 273–297
Grim P., Selinger E., Braynen W., Rosenberger R., Au R., Louie N., Connolly J. (2004) Reducing prejudice: A spatialized game-theoretic model for the contact hypothesis. In: Pollack J., Bedau M., Husbands P., Ikegami T., Watson R. A. (eds) Artificial life IX. MA: MIT Press, Cambridge, pp 244–249
Grim P., Selinger E., Braynen W., Rosenberger R., Au R., Louie N., Connolly J. (2005) Modeling prejudice reduction: Spatialized game theory and the contact hypothesis. Public Affairs Quarterly 19: 95–125
Guala F. (2002) Models, simulations, and experiments. In: Magnani L., Nersessian N. (eds) Model-based reasoning: Science, technology, values. Kluwer, New York, NY, pp 59–74
Gumerman G. J., Swedland A. C., Dean J. S., Epstein J. M. (2003) The evolution of social behavior in the prehistoric American Southwest. Artificial Life 9: 435–444
Huberman B., Glance N. (1993) Evolutionary games and computer simulations. Proceedings of the National Academy of Science, USA 90: 7716–7718
Huggins E. M., Schultz E. A. (1967) San Francisco bay in a warehouse. Journal of the Institute of Environmental Sciences and Technology 10(5): 9–16
Huggins, E. M., & Schultz, E. A. (1973). The San Francisco bay and delta model. California Engineer, 51(3), 11–23. http://www.spn.usace.army.mil/bmvc/bmjourney/the_model/history.html.
Interagency Performance Evaluation Task Force. (2006). Performance evaluation of the New Orleans and Southeast Louisiana Hurricane Protection System: Draft final report of the Interagency Performance Evaluation Task Force (Vol. 1). www.asce.org/files/pdf/executivesummary_v20i.pdf.
Kitcher P. (1993) The evolution of human altruism. Journal of Philosophy 90: 497–516
Küppers G., Lenhard J. (2006) From hierarchical to network-like integration: A revolution of modeling style in computer-simulation. In: Küppers G., Lenhard J., Shinn T. (eds) Simulation: Pragmatic construction of reality. Springer, Dordrecht, pp 89–106
Mayo, D., Hollander, R. D. (eds) (1994) Acceptable evidence: Science and values in risk management. University Press, New York, NY
Morgan M. (2003) Experiments without material intervention: Model experiments, virtual experiments and virtually experiments. In: Radder H. (eds) The philosophy of scientific experimentation. University of Pittsburgh Press, Pittsburgh, pp 216–235
Ngenkaew, W., Ono, S., & Nakayama, S. (2007). Multiple pheromone deposition in ant-based clustering as an ant foraging concept. In S. Sahni (Ed.), Proceedings of the 3rd IASTED International Conference, Advances in computer science and technology (pp. 432–436). Anaheim, CA: Acta Press.)
Nowak M., May R. (1993) The spatial dimensions of evolution. International Journal of Bifurcation and Chaos 3: 35–78
Nowak M., Sigmund K. (1992) Tit For Tat in heterogeneous populations. Nature 355: 250–252
Parker W. S. (2008) Computer simulation through an error-statistical lens. Synthese 163(3): 371–384
Parker W. S. (2009) Does matter really matter? Computer simulations, experiments, and reality. Synthese 169: 483–496
Pearl J. (2000) Causality: Models, reasoning, and inference. Cambridge University Press, Cambridge, MA
Rescher N. (1998) Predicting the future. SUNY Press, Albany, NY
Resnick M. (1997) Turtles, termites, and traffic jams: Explorations in massively parallel microworlds. MIT Press, Cambridge, MA
Robinson, M. C. (1992). Rivers in miniature: The Mississippi Basin Model. In B. W. Fowle (Ed.), Builders and fighters: U.S. Army Engineers in World War II. Fort Belvoir, VA: Office of History, United States Army Corps of Engineers.
Schelling T. C. (1978) Micromotives and macrobehavior. Norton, New York, NY
Smith J. M. (1995) Life at the edge of chaos?. New York Review of Books 42(4): 28–30
Spirtes P., Glymour C., Scheines R. (2000) Causation, prediction, and search (2nd ed.). MIT Press, Cambridge, MA
Sterrett S. G. (2005) Wittgenstein flies a kite. Pi Press, New York, NY
Suarez M. (2003) Scientific representation: Against similarity and isomorphism. International Studies in the Philosophy of Science 7: 225–244
Suarez, M. (eds) (2009) Fictions in science: Philosophical essays on modeling and idealization. Routledge, New York, NY
Suppes P. (2002) Representation and invariance of scientific structures. CSLI Publications, Stanford, CA
Teller P. (2001) Twilight of the perfect model. Erkenntnis 55: 393–415
van Fraassen B. C. (1980) The scientific image. Oxford University Press, Oxford
van Fraassen B. C. (2008) Scientific representation: Paradoxes of perspective. Oxford University Press, New York, NY
Walton K. (1990) Mimesis as make-believe. Harvard University Press, Cambridge, MA
Winsberg E. (2003) Simulated experiments: Methodology for a virtual world. Philosophy of Science 70: 105–125
Winsberg E. (2009) A tale of two methods. Synthese 169: 575–592
Woodward J. B. (2003) Making things happen: A theory of causal explanation. Oxford University Press, New York, NY
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Grim, P., Rosenberger, R., Rosenfeld, A. et al. How simulations fail. Synthese 190, 2367–2390 (2013). https://doi.org/10.1007/s11229-011-9976-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11229-011-9976-7