At an open workshop entitled “The Conceptual Basis of Biology” held in Washington DC recently (Board of Life Sciences 2008), Nancy Knowlton, a marine ecologist and conservation biologist at Scripps Institution of Oceanography in La Jolla, CA, observed and lamented that community ecology did not possess a general theory comparable to evolutionary biology. If the explanatory target of evolutionary biology is understood to be the explanation of diversity and adaptation, then the central narrative of evolutionary theory is that variation originates from random mutation and then natural selection in a local setting acts upon this variation to produce organic diversity. No similarly general narrative exists for community ecology, a subject whose explanatiory target is to explain the distribution and abundance of organisms, assuming that the characteristics (phenotypes) of those organisms can be taken as given. Instead, theoretical community ecology consists of many models about ideas, phenomena and particular classes of systems that do not spring from a common narrative. Professor Knowlton further observed that the turnover of models was faster in ecology than in evolution. In recent decades, we have seen concepts like r- and K-selection, niche theory, the keystone predator, the intermediate disturbance principle, gap-structured forests etc. launched as general organizing principles in community ecology and then sink to some modest pedestal in the pantheon of ecological concepts. In contrast, Professor Knowlton suggested that over a similar period of time, only group selection has disappeared from prominence in evolutionary thinking. The central narrative of evolutionary biology has remained relatively permanent while theoretical research has extended or broadened that narrative.

Those of us who have contributed to ecological community theory since the 1970s perceive a cycle of interest in a general theory. I came of age as a theoretical ecologist when the goal of a general theory for community ecology drove our research, a goal propounded initially by Hutchinson (1959), and developed by MacArthur and Levins (1967), and by May (1973) into a body of mathematical theory and attendant ideas called niche theory. The idea behind niche theory is that interspecific competition causes species to persist in different niches, resulting in a bookcase metaphor for community structure. All communities were envisioned as books stored on a shelf of given length. The books were allowed to overlap a bit on the shelves (niche overlap) and to have varying thicknesses (niche width). A community’s diversity then emerged as the number of books that can be shelved on the book case. The impact on diversity of say, global change or other policy actions could then be predicted by assaying how such actions affected the bookcase length, book width and allowable book overlap and then computing the new diversity that results. Many of us worked on developing this metaphor into a mathematical model based on the Lotka-Volterra competition equations (Roughgarden 1972, 1976). Although there was controversy about whether competition between species was as widespread as this metaphor required, subsequent empirical work did confirm a ubiquity of interspecific competition (Diamond 1973; Brown 1975; Schoener 1982, 1984; Pacala and Roughgarden 1985), but also disclosed a near absence of any bookcase-like general plan of organization for communities. While niche-spacing could be found in certain guilds, such findings assumed the status of neat special cases rather than as instances of a general pattern of community structure.

The reaction of empirical community ecologists as well as many theoreticians during the 1980s and 1990s then became one of skepticism, even hostility, to general theory. General theory was viewed as a top-down imposition of theoretical claims on the reality of ecological communities, and models that were tailored for particular classes of communities were demanded instead. Less well known, but of even more concern to theoreticians, was the discovery by the mathematician, Stephen Smale, of how sensitive the predictions of non-linear dynamical models are to fine details in the assumptions (Smale 1976). If the predictions from such models, including the Lotka-Volterra equations, are fragile, one needs empirical confidence in whatever model formulation is used before investing effort at mathematical analysis. So, during the 1980s and 1990s the attention of theoreticians in community ecology turned to representations of particular classes of systems, such as lakes (Carpenter et al. 1985), the rocky intertidal zone (Roughgarden et al. 1985; Roughgarden and Iwasa 1986), forests (Pacala and Silander 1985), and so forth (Roughgarden 1998). While generality was abandoned, one had in its place a sense that the models were actually relevant to real systems—it is better to have a special theory about something than a general theory about nothing.

Also in the 1990s, the notion of an ecological community as a biological system unto itself came up for increasing reexamination. For many years the ecosystem concept has treated an ecological system as inherently comprised of both biotic and abiotic components in equal measure—the ecosystem is the ecological community (biotic part) plus the watershed, soil, and local atmosphere etc. (the abiotic parts) and considers them a combined interacting system. The focus of ecosystem ecology has been biogeochemistry, rather than species composition. In the 1990s population and community ecologists similarly realized that population dynamics and community composition were determined as much by physical transport processes as by biotic interactions. My own work along the California coast showed that the transport resulting from upwelling features in coastal circulation determines intertidal population dynamics and community composition as much or more than biotic interactions of competition, predation and mutualism (Roughgarden et al. 1988, 1991, 1995; Gaines and Roughgarden 1985; Shkedy and Roughgarden 1997; Connolly et al. 2001). I also found for islands in the Caribbean that the community composition there depends as much on plate-tectonic motion as on the biotic interactions among residents (Roughgarden 1990, 1995). This realization came to be known as “supply-side ecology” (Roughgarden et al. 1987), and implied that community ecology theory, like ecosystem theory, needed to include both abotic and biotic components in equal measure. But instead of focusing on biogeochemistry, community ecology’s focus is on what we might term “biogeotransport”. The composition of an ecological community, and the coupling between spatially distant ecological communities, is thus effected by animals (including animals that disperse plants) who provide their own locomotion, animals and plants who act as smart sailors and clever pilots to navigate the ocean and atmospheric currents, and species who are moved fortuitously by plate-tectonic transport.

The theoretical consequence of viewing ecological communities as comprised in equal measure of biotic and abiotic components is that community ecology is no longer part of biology as traditionally conceived. Instead, community ecology becomes an “organic earth science”, applying methods and perspectives of the physical earth sciences to ecological systems, and is part of “earth-systems science”. This realization affects the mathematical modeling of communities because now purely biotic approaches, such as a bookshelf of niches, or food web topologies, need to be supplemented with state variables or nodes that refer to features of the physical environment in addition to those for various species.

Furthermore, if community ecology is really an organic earth science as much as a biological science, then the possibility of any general theory for what the community structure happens to be at each spot in the world would seem doubtful just as there is no general theory for what the pattern of diversity turns out to be at each spot in the world. After all, there is no general theory of geology. No mathematical model predicts India’s collision with Asia to produce the Himalayas, or Pangaea’s instability and break up, or predicts where and when an earthquake will appear. Fusing community ecology with earth science combines the idiosyncrasy of species identities with the idiosyncrasy of geological place. A general model of the rocky intertidal zone would await a general model of coastal circulation, which is not on the horizon, and a general model of desert communities would await a general model of desert hydrography, which is not on the horizon either.

The idea that community ecology is an organic earth science as much as a biological science finds antecedents in the geographic classification of biomes. Even the late theoretician, Robert MacArthur together with the marine ecologist, Joseph Connell, produced an undergraduate textbook with a two-page foldout presenting the major community types located according to geographic characteristics (MacArthur and Connell 1967). Robert MacArthur’s last book was entitled Geographical Ecology (MacArthur 1984) which suggests a perspective that every community is unique to time and place. As such, the structure of ecological theory winds up being an atlas of models that work in various geographic locations rather than a general set of propositions that refers to all communities.

Nonetheless, as we are well into the decade of the 2000s it is clear that interest in a general theory of community ecology has resurfaced. A distinguished empirical ecologist like Nancy Knowlton is voicing a call for a general theory of ecological communities, rather than, as might have happened in the past, rejecting any attempt at general theory as a distorting imposition of theoretical artificialities. So, let us contemplate, in light of nearly four decades of experience with theoretical community ecology, whether a general theory is possible, and if so, how we might go about it, and whether, after examination, we still think this is a good idea.

The early intellectual history of ecology differs from evolutionary biology. Darwin’s theory is about the production of diversity, not about patterns of diversity. It doesn’t take a position on whether all of life’s diversity can be boiled down to rule-like general patterns, although patterns do occur like adaptive radiation, convergent and parallel evolution, and so forth. Instead, evolutionary theory accepts whatever the diversity is to begin with and then explains how it comes about. In contrast, ecological community theory has always focused first on structure, and secondarily about how a community is produced.

In 1916, the plant ecologist, Clements advanced the metaphor of the community as a super “organism” containing species whose activities combine to achieve integrated functions (Clements 1916). Then in the 1920s, Gleason, another botanist, advanced the “individualist” concept of a plant community and argued that the plant species in a community served no common function but pursued private agendas (Gleason 1926). Both positions start with what a community is, and then offer a theory of formation: Clements views community formation as succession from pioneer to mature climax analagous to organismal development, whereas Gleason views community formation as an accumulation of independent dispersal events for various plant species.

This early focus on community structure first, and community formation second, continues into the present. MacArthur and Levin’s niche theory, as already mentioned, offered a view of community structure as a bookcase of species. MacArthur and Wilson’s island-biogeography theory presented species-area patterns (MacArthur and Wilson 1963). Preston introduced the truncated log-normal curve for the pattern relating abundance to diversity (Preston 1962). Odum’s energy-flow theory replaced the pyramid of number with a pyramid of trophic energy stocks and accounted for the pyramidal shape with the energetic transfer efficiencies derived in part from thermodynamics (Odum 1969). Cohen’s surveys of food webs have sought topological generalizations about food web graphs and have reported statistics like average chain lengths and average number of tropic levels (Cohen 1978). Tilman’s research with plant communities has recast the Lotka-Volterra equations of niche theory into equations for consumer-resource dynamics (Tilman 1982). More recently, Brown and colleague’s theory of scaling has revealed patterns of body size and metabolism broadly shared by many organisms (Brown et al. 2004), Hubbell’s “neutral theory” of species diversity has shown how many diversity patterns can be derived from genetic-drift-like combinatorics (Hubbell 2001), and Harte and colleagues have shown that spatial diversity patterns can be derived from statistical-mechanical-like combinatorics (Harte et al. 1999). All of these research projects have been successful, at least to some extent, and some are on-going. They show what I think are commonalities rather than generalizations.

A long look at ecological communities does reveal many statistical regularities. Yet, there is no theoretical necessity that any instance of a community should coincide with any of the reported regularities, and all the reported regularities enjoy ample exceptions, which some people downplay and others savor. Do the broad commonalities add up to a “general theory” of community ecology? Most people probably don’t think so (Lawton 1999; Simberloff 2004). The commonalities don’t have the linguistic form of, “All communities have property X”. Instead, they have the form of, “Most communities of type Y have property X, except when Z happens”. Nor do they have the even stronger form of, “If A is an ecological community, then it must have property X”, which a genuine generalization would have. So, how are we to find a general theory of community ecology similar to that of evolutionary biology?

The answer, I suggest, is to focus on a formational rather than structural theory for community ecology. Consider evolution’s central narrative: natural selection in local situations acts upon random mutations to produce a diversity of traits. A logical counterpart can be stated for community ecology as well. First, let’s agree to define a community as all living organisms in a prescribed area. Then we can say: local interactions act upon the species arriving at the community’s boundary to produce a diversity of communities. This is a formational theory, and takes no position before hand on the structural or functional characteristics of the ecological communities that result. Such a theory would consist of a supply side and an interaction side—the supply side is the logical counterpart of mutation, and the interaction side is the logical counterpart of natural selection. Community ecology is not materially analogous to evolution—that is, there is no selection at the community or any other higher level. Instead, a formational theory of ecological communities is logically parallel to the theory of evolution. Dispersal and physical transport processes supply the species upon which a matrix of interactions that includes predation, competition, mutualism and coevolution acts to determine which arriving species successfully enter the community and which species disappear thereafter or change in abundance. This proposition leads to a general theory for the diversity of communities, not for the diversity in communities. Then properties within a community such as its species composition and dynamic stability emerge as a special case, or class of cases, that depends on local situation. Yet all communities, each and every one of them, result from species somehow arriving at their boundaries, and then being denied or welcomed entry by local interactions. Therefore, this narrative of community formation is inherently general and the story for every particular community is logically subordinate to this general theory.

I thank the Editor for observing how my thoughts about general theory in ecology are consistent with recent discussion in the philosophy of science by Woodward, among others (Woodward 2003, 2002, 2001). Apparently, the sciences rarely deliver exceptionless generalizations and scientific effort instead may be directed toward finding an “invariant toolkit” of mechanisms that yield a variety of outcomes resulting from different interplays among the mechanisms in various situations.

What are the pros and cons of framing community ecology as a general theory? Consider the advantages first. The knowledge of community ecology could be organized in a common format that embraces the joint contribution of both earth science processes and biological-population processes to producing the diversity of communities. Furthermore, because models of communities could be formulated with a common template, general theorems could be derived pertaining to all communities, theorems that might allow any particular community’s structure and function to be partitioned into the contributions coming from the supply side and those coming from the interaction side. And general formulas might be derived for what might be thought of as the “heartbeat” of a community which would indicate whether a community is near collapse. Indeed, the prospect of taking a community’s “pulse” is attractive from a conservation perspective, and would allow for an inventory of the state of “health” of the ecological communities around the world. Nancy Knowlton’s call for general theory was prompted by alarm that the coral reefs where her research takes place are endangered and that no one knows how to approach determining how pressing the crisis is or how to deal with the crisis most economically.

But there are disadvantages to framing community ecology as a general theory too. The low turnover of theoretical ideas in evolution relative to ecology that Nancy Knowlton noted does not imply that all of evolutionary theory is indeed correct, but may instead imply that dissonant voices and hypotheses are carefully policed and squelched. As a San Francisco lefty, the radical spirit in me advises caution about general theory, as I often sense the motivation for generality is to control rather than understand, to dominate rather than appreciate. Do we want to arm scientists with a general theory of communities with which they might someday re-engineer the communities of the world, even if those scientists presently calling for a general theory have the purest of motives? And a general theory can readily attain establishment status, maintained by university curriculum committees, department structures, and funding-agency peer-review systems, and then defended with claims of scientific consensus, consensus that later turns out to be mistaken. The interlocking claims in a general theory also brings the danger that an entire theoretical system can continue on the wrong track, with a momentum of its own that crushes any contrarians in its path. One need only observe how alternatives to the theory of sexual selection in evolution, the area where my present research is concentrated, are rebuffed with vitriol (Roughgarden 2007) despite ample contrary evidence, logical flaws, internal inconsistencies, and a pejorative, value-loaded vocabulary (Roughgarden 2009). In contrast, new ideas and approaches find safety and space in a community ecology whose theory remains disaggregated. So, I find I’m comfortable with theoretical community ecology as an atlas of models.

Still, I appreciate the appeal of framing theoretical community ecology as a general theory. Perhaps if we’re cautious, if we search for a general theory with our eyes open, we can avoid the dangers that attend a general theory. So, is there a general theory of community ecology today? No. Can there be? Yes. Should there be? I don’t know.