Keywords

12.1 Systematic and Phylogenetic Context

This chapter focuses on the early phases of brain evolution in the order Primates, with only a brief discussion (Sect. 12.6) of evolutionary events occurring higher in the primate tree. Therefore, this section is largely focused on the taxa (and taxonomic framework) most relevant to that perspective.

12.1.1 The Phylogenetic Position of Primates Within Mammalia

Identifying the mammalian orders most closely related to Primates is central to providing a context for studying primate brain evolution, particularly when considering the earliest phases of this process. Historically there were two main hypotheses about the closest relatives to Primates. First, an ancestry among “insectivores” (i.e. shrews, moles, hedgehogs, desmans, solenodons, and historically golden moles and tenrecs [now considered afrotheres]) has long been posited for the order (e.g. Simpson 1945; McKenna 1966; Szalay 1975). In particular, the general dental similarities with erinaceomorphs (i.e. hedgehogs) suggested to some workers that primates may have arisen from among this group or shared a common ancestor with it (see discussion in MacPhee et al. 1988). With respect to the evolution of the brain, this suggested link formed part of the basis for comparisons between living “insectivores” and Primates in the classic compilation of volumetric data by Stephan and colleagues (Stephan et al. 1970, 1981). These authors also posited that extant insectivores formed a good general model for the primitive form of the brain, and in particular identified a subset of taxa (shrews and hedgehogs) as showing what they inferred to be relatively primitive cerebral patterns. This dataset formed the basis for a series of publications focusing on the evolution of different regions of the brain (Stephan 1972), such as the neocortex (e.g. Frahm et al. 1982), in a framework that was explicitly rooted in “insectivores” as models for what was primitive for Primates. These works played a central role in framing ideas about early transitions in the size and form of the brain around the origin of the order (see for example Martin 1990).

Second, the alternative perspective, dating back to Gregory (1910), was that Primates were most closely related to treeshrews (Scandentia), elephant shrews (Macroscelididae), colugos (Dermoptera) and bats (Chiroptera), with these various orders being grouped with Primates in Archonta. Unpopular for several decades after its proposal, this idea was re-vivified starting in the 1970s, based on a version of Archonta that excluded elephant shrews (e.g. McKenna 1975; Szalay 1977). Although treeshrews (often as putative primitive primates) were included in early discussions of the evolution of the brain in Primates (e.g. Elliot Smith 1902; Le Gros Clark 1945; Stephan et al. 1970, 1981; Martin 1973), a perspective that considered Archonta as the critical comparative context rather than “Insectivora” was absent.

Molecular analyses of mammalian inter-ordinal relationships have led to a broad-based consensus about which taxa should be considered Primates’ closest kin (Fig. 12.1). There is strong support for a modified version of Archonta (i.e. Euarchonta Waddell et al. 1999) that includes Primates, Scandentia, and Dermoptera, but not Chiroptera. Within Euarchonta there is some lingering debate about which order(s) is the sister taxon of Primates, with there being analyses supporting all possible resolutions (i.e. Dermoptera, e.g. Janečka et al. 2007; Scandentia, e.g., Liu et al. 2009; or Sundatheria [Dermoptera + Scandentia], e.g. O’Leary et al. 2013). Recent genomic analyses seem to support a resolution to this debate, with Dermoptera being Primates’ sister group (Mason et al. 2016; Zhang et al. 2019). The closest relatives of Euarchonta are not “insectivores” but rather rodents, rabbits and pikas (i.e. Glires [Rodentia + Lagomorpha]), a relationship recognized by the supraordinal name Euarchontoglires (Murphy et al. 2001). “Insectivora” as historically conceptualized is no longer considered to be a valid grouping; instead, supposed “insectivores” are thought to be spread between two broadly divergent supraordinal groups, the endemic African Afrotheria (Stanhope et al. 1998) and the more northerly evolving Laurasiatheria (Murphy et al. 2001). Hedgehogs in particular are included in Eulipotyphla, which is part of Laurasiatheria, and as such are more closely related to bats, carnivores, and ungulates than they are to Primates (Murphy et al. 2001).

Fig. 12.1
figure 1

Hypothesis of relationships for members of Euarchontoglires discussed in this chapter, based largely on Silcox et al. (2010b). Dermoptera has been positioned as the sister taxon to Primates based on Mason et al. (2016)

Although this phylogenetic framework is broadly agreed upon, lingering effects of the history of considering “insectivores” as relevant to establishing what is primitive for Primates remain, with analyses as recent as 2016 (e.g. Ni et al. 2016) still including hedgehogs as outgroups to Euarchonta, rather than members of Glires (see also Beaudet and Gilissen 2018). This is also true for considerations of brain evolution (e.g. Gingerich and Gunnell 2005), so that even in our own work (Silcox et al. 2009b, 2010a), “insectivores” were used as proxies for what is primitive in Primates, in the absence of better available options.

With respect to the paleoneurological record, part of the challenge with studying the early evolution of the brain in Primates is that there are no fossil crania of Scandentia or Dermoptera that are complete enough to produce an endocast for the purposes of comparison. As noted above, data from living treeshrews have been incorporated, to some degree, into discussions of primate brain evolution, and there exist very detailed histological descriptions of the modern treeshrew brain in a small selection of species (e.g. Tupaia glis, Tigges and Shantha 1969; Tupaia belangeri, Zhou and Ni 2016), as well as a database of endocasts for a greater diversity of extant forms (San Martin-Flores et al. 2018). However, based on comparisons to early primates, modern treeshrews make a poor proxy for a primitive stage of primate brain evolution, likely as a result of parallel increases in some areas of the brain (e.g., the neocortex; San Martin-Flores et al. 2018). Dermopterans, who have encephalization quotients (EQ) that are low relative to those of living Primates (Gingerich and Gunnell 2005), nonetheless have gyrencephalic brains that are very different from what would be expected for a primitive primate (San Martin-Flores et al. 2019).

From a paleoneurological perspective, this makes the endocasts of fossil Glires very relevant to studying primitive states in Primates, as the only extant group of non-primate euarchontoglirans for which well-preserved fossil crania are known. Meng et al. (2003) published natural endocasts of the primitive member of Glires Rhombomylus turpanensis, although unfortunately they did not provide any quantitative data. There is a growing record of endocasts for fossil rodents (e.g. Dechaseaux 1958; Dozo 1997a, b; Dozo et al. 2004; Bertrand and Silcox 2016; Bertrand et al. 2016, 2017, 2018, 2019a; Ferreira et al. 2020), including some fairly primitive taxa (i.e ischyromyids; see Bertrand and Silcox, Chap. 16, this book). Less well known is the form of the brain in extinct members of Lagomorpha, with Cope (1884) providing a few details about a natural endocast of Paleolagus, but otherwise only a few natural endocasts for relatively recent specimens being available (Edinger 1929; Sych 1967; Czyżewska 1985). More recently, virtual endocasts for extant lagomorphs and one virtual endocast for a more basal member of that order (Megalagus turgidus; López-Torres et al. 2020) have been described. Although still limited, the record that is available for Rodentia and Lagomorpha does help to frame primitive states for Primates, as discussed below.

Also, potentially relevant to assessing the primitive form of the brain in Primates are extinct groups that have been inferred to be members of Euarchontoglires (e.g. Apatemyidae [Silcox et al. 2010b], Anagalidae [Meng 2004], Mixodectidae [Szalay and Lucas 1996; Sargis et al. 2018]). Of these, the Apatemyidae is notable because virtual endocasts have been published for two species (see discussion below; von Koenigswald et al. 2009; Silcox et al. 2011). Apatemyids were arboreal animals (von Koenigswald 1990; von Koenigswald et al. 2005) sharing some features in the postcranium with euarchontans (Bloch et al. 2004), and with similarities to plesiadapiforms in the presence of enlarged, procumbent upper and lower incisors (e.g. see Silcox et al. 2010b: fig 2). An analysis based on craniodental traits grouped them within Euarchontoglires, with weak support tying them to Rhombomylus (Silcox et al. 2010b). As such, they have been suggested to be relevant to the larger context of euarchontogliran brain evolution (Silcox et al. 2011).

12.1.2 Taxonomy and Phylogeny of Primates

For extant primates, there is a broad-based consensus on the major framework for relationships within the order (e.g. Springer et al. 2012; Fleagle 2013). The first major division into suborders is between Strepsirrhini (lorises, lemurs and galagos) and Haplorhini (tarsiers, monkeys, apes and humans). Within Haplorhini, tarsiers are considered the most basally divergent group; their behavioral and morphological similarities with some strepsirrhines (e.g., nocturnal activity period; vertical clinging and leaping locomotion; faunivorous diet etc.) had traditionally caused them to be allied with strepsirrhines in Prosimii (engendering the term “prosimian”, which is still in broad usage), but those similarities are now thought to be primitive or convergent. The group that includes all non-tarsiiform haplorhines is variously referred to as Anthropoidea or Simiiformes. It is divided into Platyrrhini (Panamerican monkeys) and Catarrhini (apes and humans [Hominoidea] and Afroeurasian monkeys [Cercopithecoidea]).

Although this phylogenetic and taxonomic framework is nearly universally accepted for living primates, fitting fossil taxa into the picture is not always straightforward, particularly for primitive species. The oldest potential primates are part of a radiation of over 140 species in 11 families that are generally referred to as plesiadapiforms (Silcox et al. 2017a). The first plesiadapiforms appear not long after the non-avian dinosaurs went extinct, in the early Paleocene (Fox and Scott 2011; Wilson Mantilla et al. 2021), whereas the latest occurring plesiadapiforms are late Eocene in age (Kihm and Tornow 2014). In the intervening >27 million years, members of the group evolved an impressive diversity of adaptations, although all known species have enlarged upper and lower central incisors and all species known from postcranial material were non-leaping arborealists. The primate status of plesiadapiforms continues to be a matter of debate. Whereas they share similarities to living primates in aspects of the dentition (e.g., low-crowned, bunodont molars with broad talonid basins) and in adaptations of the postcranium for arboreality, plesiadapiforms lack some traits that have traditionally been considered important to identifying primates, such as the postorbital bar. In recent years, the continuation of the debate stems in part from the challenge of choosing between the results of cladistic analyses based on larger matrices that were not designed with plesiadapiform character states in mind (e.g. Ni et al. 2016), and smaller matrices that were more explicitly tailored to the problem of sorting out events near the base of the primate tree (e.g. Bloch et al. 2007; Silcox 2008; Silcox et al. 2010b; Chester et al. 2017, 2019; see discussion in Silcox et al. 2017a). In the current paper we consider plesiadapiforms to be stem primates—so members of the order, but without a particular tie to any modern groups (Fig. 12.1). It is worth noting, however, that even analyses that come to a divergent conclusion about their primate status still finds that they are members of Euarchonta (e.g. Ni et al. 2016). As such, they are relevant to assessing primitive states for Primates whether or not they are classified as such. Within this framework it is useful to make a distinction between Plesiadapiformes, as a likely paraphyletic array of stem primate families, and Euprimates Hoffstetter, 1977, as (probable) crown primates (Fig. 12.1).

The other two groups that are particularly critical for studying early brain evolution in Primates are Adapoidea and Omomyoidea, extinct euprimate superfamilies that both appear in the earliest Eocene (approx. 56 mya; Ni et al. 2004; Smith et al. 2006; Beard 2008; Rose et al. 2011, 2012). Most workers would agree that omomyoids are probably related to tarsiiforms, or at least are haplorhines (e.g. Ni et al. 2016), but relationships of adapoids are more controversial, with various authors putting them on different sides of the haplorhine/strepsirrhine split (e.g. Gingerich et al. 2010; Williams et al. 2010). The consensus leans towards considering them strepsirrhines, in part because that is where they fall out in all large scale cladistic analyses (e.g. Ni et al. 2016; Seiffert et al. 2018). However, it is worth noting that they lack traits such as the toothcomb that are often thought to be distinctive of strepsirrhines (e.g. Fleagle 2013), implying that they are at best stem strepsirrhines. With respect to the paleoneurological record, adapoids and omomyoids are critically important, because there are no endocasts of early crown strepsirrhines (the oldest being the natural endocast of the Miocene lorisiform Komba; Le Gros Clark and Thomas 1952; Simpson 1967), or other early, non-anthropoid haplorhines, but there is a burgeoning record of endocasts for adapoids and omomyoids.

12.2 Historical Background

12.2.1 The Record of Endocranial Morphology and Any Other Paleoneurological Approaches in the Group Under Study

There is a long history of study for endocasts of fossil primates, likely motivated by an interest in situating the exceptionally large brains of humans in a broader evolutionary context. The discussion below is divided into “Pre-CT” and “Post-CT” because the widespread availability of high-resolution X-ray computed tomography has re-framed the type of data that can be extracted from fossil primate crania.

12.2.1.1 Pre-CT

Discussion of the paleoneurology of early primates extends back to at least 1884, when Cope (1884, 1885) provided some brief commentary on the apparent form of the brain from the cranium of “Anaptomorphus” (now considered Tetonius) homunculus. Critical references in the early study of primate endocasts include Neumayer (1906), Gregory (1920), Le Gros Clark (1945), Hürzeler (1948), Piveteau (1958), Hofer (1962), Gazin (1965), Hofer and Wilson (1967), Radinsky (1967, 1970, 1974, 1975, 1977, 1982), Szalay (1969), Jerison (1973, 1979), Gingerich (1976), Gingerich and Martin (1981), Gurche (1982), Martin (1990), and Gingerich and Gunnell (2005). Gurche (1982) published a useful summary of the state of knowledge known at the time for endocranial data of early primates, which includes consideration of most of the data available pre-CT. Although he deemed the sample available at that point to be “disappointingly small” (p. 227), he nonetheless provided a compilation of volume estimates for six species: the adapoids Smilodectes gracilis, Adapis parisiensis, and Notharctus tenebrosus; the omomyoids Necrolemur antiquus and Tetonius homunculus; and the taxonomically controversial Rooneyia viejaensis (often considered an omomyoid, but see Rosenberger et al. 2008). Prior to 1982, there were also published estimates of endocranial volume for the plesiadapiform Plesiadapis tricuspidens (Gingerich 1976; Radinsky 1977) that Gurche did not include, presumably because they were based on “the external appearance of crushed skulls” (p. 235). Of the specimens available in 1982, the most complete are attributed to the adapoids Smilodectes gracilis, known from a fairly complete natural endocast (USNM 23276; but missing the olfactory bulbs) published with excellent illustrations by Gazin (1965); and Adapis parisiensis, known from two endocasts, and for which direct estimates of volume could be calculated using both glass beads and mustard seed (Le Gros Clark 1945; Martin 1973, 1980; Gingerich and Martin 1981). While not discussed in any detail by Gurche (1982), there was also a partial latex endocast published for the microsyopid plesiadapiform Megadelphus lundeliusi (AMNH 55284) by Szalay (1969; see also Radinsky 1977), although he did not provide any associated quantitative data. All the other endocranial data had to be gleaned from partial natural endocasts still partly or largely entombed in the crania or estimated from external cranial dimensions.

The interpretation of the data from this array of specimens was the focus of a historic debate in the literature between Leonard Radinsky and Harry Jerison (Radinsky 1970, 1977, 1982; Jerison 1973, 1979). Key areas of disagreement included (1) varying estimates of the endocranial capacity for the euprimate specimens; (2) differing interpretations about what the available data for Plesiadapis could tell us about the very earliest phases of primate brain evolution (i.e., with respect to the size of the brain and the degree to which it could be considered “spheroidal” like a primate’s); (3) differences of opinion over how to assess relative brain size (i.e. based on varying body mass estimators, and the use of different proxies for body mass such as foramen magnum dimensions); and (4) divergent views about the appropriate comparative context (i.e., modern primates vs. contemporary fossil taxa). Ultimately, the central difference of opinion between these authors was whether or not the evidence was adequate to assert that “encephalization was probably a characteristic adaptation in the order Primates from the earliest times,” (Jerison 1979: 615), with Radinsky (1977, 1982) disagreeing with this perspective. Gurche’s (1982) reassessment of the relevant data (including his own set of volume estimates) concluded, that, apart from Rooneyia, the Eocene euprimates had small brains relative to those of modern prosimians, with the adapoids in particular being notably less encephalized.

From Gurche’s (1982) summary to the beginning of the CT era, additional data for only three early Tertiary fossil primate species were added to the picture: the adapoids Leptadapis magnus and Pronycticebus gaudryi (Martin 1990; note that the endocast referred to as “Adapis magnus” by Piveteau, 1958, actually pertains to A. parisiensis [Gingerich and Martin 1981]), and the plesiadapiform Plesiadapis cookei (Gingerich and Gunnell 2005). The latter was extremely revelatory with respect to the earlier arguments about the size and form of the brain in Plesiadapis. Gingerich and Gunnell (2005) made an estimate of cranial capacity using a full-scale model based on a partial natural endocast, and on dimensions drawn from a fairly completely preserved skull roof. Although the dorsoventral depth had to be approximated from “comparison with a range of endocasts of similar living mammals” (p. 188), this calculation is nonetheless much better constrained than earlier attempts to estimate the form and volume of the brain in the closely related species P. tricuspidens (Gingerich 1976; Radinsky 1977; Jerison 1979). The endocranial volume measured was much, much smaller than estimated for the similarly sized P. tricuspidens (i.e. 5 cc for P. cookei compared to estimates of 18.6 cc [Gingerich 1976]; 12–17 cc [Radinsky 1977]; and 16.6 cc [Martin 1990] for P. tricuspidens), and the shape of the endocast was far from spheroidal (Gingerich and Gunnell 2005: fig. 3). An excellent estimate of body mass can also be made for this specimen (UM 87990) because it is associated with much of a skeleton (Gingerich and Gunnell 2005; Boyer and Gingerich 2019). The ultimate message from these analyses is that P. cookei had a brain that was relatively very small compared to living primates and living dermopterans, and actually within the range of variation for Paleocene archaic ungulates. These data provided a first suggestion that Jerison’s generalization about encephalization being an ancient trait for Primates may not hold for “the first evolutionary radiation of primates” (Radinsky 1982: p. 34).

12.2.1.2 Post-CT

The small size and fragility of the cranium in most primitive primates limited the data available from traditional approaches. The increasing availability of high-resolution X-ray CT data has begun to revolutionize our understanding of their endocranial anatomy, particularly with respect to gathering accurate quantitative data. Virtual endocasts have been published for plesiadapiforms from three families: Plesiadapidae, Paromomyidae, and Microsyopidae (Fig. 12.2; Silcox et al. 2009b, 2010a; Orliac et al. 2014; White et al. 2016). With respect to adapoids and omomyoids, virtual endocasts have been published for many of the same species whose significance was debated by Radinsky, Jerison, and Gurche, including Smilodectes gracilis, Adapis parisiensis, Notharctus tenebrosus, Rooneyia viejaensis, and Necrolemur antiquus (Fig. 12.3; Kirk et al. 2014; Harrington et al. 2016, 2020); notably Harrington et al. (2016) were able to provide endocasts for multiple specimens of N. tenebrosus (N = 3) and S. gracilis (N = 4), including a subadult specimen of S. gracilis (UM 32773 [=MPM 2612]), allowing for some first glimpses into intraspecific variation and ontogenetic change. Ramdarshan and Orliac (2016) provided a substantively complete endocast for the omomyoid Microchoerus erinaceus, a close relative of N. antiquus.

Fig. 12.2
figure 2

Virtual endocasts of fossil stem primates from the families Paromomyidae (Ignacius graybulllianus, USNM 421608), Microsyopidae (Microsyops annectens, UW 12362), and Plesiadapidae (Plesiadapis tricuspidens, MNHN CR 125) in lateral, dorsal, and ventral views. Endocasts originally published in Silcox et al. (2009b, 2010a) and Orliac et al. (2014)

Fig. 12.3
figure 3

Virtual endocasts of fossil euprimates from the superfamilies Omomyoidea (Necrolemur antiquus, MaPhQ 289 [Montauban 9]; Microchoerus erinaceus, UM-PR 1771) and Adapoidea (Notharctus tenebrosus, AMNH 127167; Smilodectes gracilis, UM 32773; Adapis parisiensis, NHM M 1345). Rooneyia viejaensis (TMM 40688-7) is of somewhat ambiguous systematic affiliation, but is often included in the Omomyoidea. Endocasts in dorsal, ventral, and lateral views. Endocasts originally published in Kirk et al. (2014), Harrington et al. (2016, 2020), and Ramdarshan and Orliac (2016)

For the taxa now known from virtual endocasts, it is possible to assess the previously made estimates of volume (see Gurche 1982: table 2; Martin 1990: table 8.12), with the assumption being that the virtual estimate is likely to be more accurate than estimates based on external dimensions or water displacement of “restored” endocasts (Gurche 1982: p. 228; Table 12.1). For Adapis parisiensis, the volume estimate made by Martin (1973) using mustard seed is a very close match to the volume calculated for the virtual endocast (8.8 cc; Harrington et al. 2016) for the same specimen, higher than Gurche’s (1982) estimate (8.31 cc), and lower than estimates calculated by Jerison and Radinsky using double integration methods (9.00 cc, 9.40 cc). Harrington et al. (2016) did not create virtual endocasts for the same specimens previously assessed for S. gracilis and N. tenebrosus, but in general their range of estimates is lower than those produced by other methods (i.e. range of 7.44–8.63 cc for S. gracilis vs. 9.12–9.95 cc [Gurche 1982]; range of 7.38–8.06 cc for N. tenebrosus vs. 10.43 cc [Gurche 1982]). Previous estimates for the only known cranium of R. viejaensis were close to the volume calculated from the virtual endocast, with Gurche (1982) actually being the closest (7.234 cc [Kirk et al. 2014] compared to 7.5 cc [Radinsky 1977]; 7.0 cc [Jerison 1979]; 7.38 cc [Gurche 1982]). Gurche (1982) also provided the endocranial volume estimate (2.65 cc) for N. antiquus that is closest to the value calculated from the digital endocast of the Montauban 9 cranium (MaPhQ 289; 2.36 cc [Harrington et al. 2020]), and markedly lower than estimates made by Radinsky (1977; 4.35 cc) and Jerison (1973, 1979, 4.20 cc,) although as Harrington et al. (2020) note, those estimates depended on composite illustrations that were based in part on other specimens (see Harrington et al. 2020: fig. 1). Bearing out the prediction made by Gingerich and Gunnell (2005), the estimate of cranial capacity for P. tricuspidens based on the virtual endocast (5.21 cc; corrected for deformation [Orliac et al. 2014]) is much lower than previous estimates for that taxon (18.6 cc [Gingerich 1976]; 12–17 cc [Radinsky 1977]; 16.6 cc [Martin 1990]), resulting in EQ estimates that overlap with that calculated for P. cookei.

Table 12.1 Compilation of data on early Tertiary primate endocasts

The virtual endocasts currently available therefore address the first two issues that drove the Jerison-Radinsky debate. First, virtual endocasts provide direct measures of volume, so they do not depend on differing methods for estimation. Incomplete or damaged specimens do still require some additional interpretation—for example, the volume for the “undeformed” endocast of P. tricuspidens calculated by Orliac et al. (2014) is still likely a bit low, because they used the endocast of Ignacius graybullianus published by Silcox et al. (2009b) as their model, which comes from a skull that is also slightly pancaked. Nonetheless, these estimates come with fewer assumptions than (for example) those based on the double integration method, which models the brain as a cylinder (Jerison 1973). Second, we now have better data not only for Plesiadapis, but for several taxa (Ignacius graybullianus, Microsyops annectens) from the primate stem, all of which make clear that early primate brains retained a lot of primitive features (see discussion below).

12.2.2 Problematics

The other two issues in the Jerison-Radinsky debate remain sources of differing opinions. The best way to make comparisons of relative brain size continues to be an issue, although Martin (1990) provided a compelling argument that foramen magnum area is a poor proxy to use for body mass because of its lack of independence from brain size. The approach most recent authors have taken (e.g. Silcox et al. 2009b, 2010a; Orliac et al. 2014; Ramdarshan and Orliac 2016) has been to calculate multiple body mass estimates using equations based on different sample populations and measurements, and correspondingly provide a range of EQ estimates. Kirk et al. (2014) did not go even that far, giving no estimate of EQ for Rooneyia (but see Harrington et al. 2016 and Table 12.S1). Differences of opinion about how to best control for body mass led to a critique (Gilbert and Jungers 2017) of one of the conclusions of the Harrington et al. (2016) analysis, specifically that changes in the organization of the brain in early euprimates preceded significant brain size increase. Gilbert and Jungers (2017) raised many valid concerns over the use of the encephalization quotient to consider relative brain size in that context. However, their approach of making narrow allometric comparisons (i.e. between taxa of like inferred body mass) was flawed in largely relying on body mass estimates for diverse taxa based on cranial length, which is problematic since plesiadapiform crania are less flexed, and have longer snouts, than euprimate crania (Bloch and Silcox 2006: fig. 28; Silcox et al. 2009a). As such, their inferences are confounded by the different scaling relationships of plesiadapiform and euprimate crania. This problem makes it difficult to assess whether their conclusion that relative brain size was notably smaller in plesiadapiforms than in early euprimates is a true signal, or a by-product of that difference (see further discussion in Sect. 12.4.2). Simply put, there is no ideal way to account for body mass in discussions of relative brain size, which means that debates about these questions are likely to continue.

The final issue in the Jerison-Radinsky debate was the appropriate comparative context in which to view the endocranial data for euprimates. In making comparisons, it is important to be clear on which question one is asking. Although differing body mass estimates make the situation somewhat murky (i.e. see discussions in Kirk et al. 2014; Ramdarshan and Orliac 2016), it does seem as though early Tertiary euprimates likely had somewhat smaller brains than living euprimates (Gurche 1982; Silcox et al. 2009b, 2010a; Harrington et al. 2016, 2020; Gilbert and Jungers 2017), with Rooneyia potentially being an exception to this generalization (Kirk et al. 2014: fig. 5; note that Necrolemur also appears to be an exception in that figure, but the endocranial volume estimate used was probably too high [Harrington et al. 2020]). Jerison (1973) suggested that there is a temporal effect on brain size, a hypothesis supported for Primates in a recent analysis by Bertrand et al. (2019a: fig. 17c) who found a significant (but rather weak) relationship between EQ estimates and geological time (p < 0.05; r2 = 0.507; Bertrand et al. 2019a: table S11). However, this perspective does not provide an answer to two questions that are critical to establishing whether or not “encephalization was…a characteristic adaptation in the order Primates from the earliest times” (Jerison 1979, p. 615).

First, it does not answer the question of whether primates were encephalized relative to other mammals from the early Tertiary. Radinsky (1982) made comparisons between ranges of EQ values he had calculated (Radinsky 1978) for archaic carnivores and ungulates and concluded that contemporaneous primates were not exceptional; as noted above, Gingerich and Gunnell (2005) reached the same conclusion for Plesiadapis cookei. However, subsequent analyses using a slightly expanded archaic sample (e.g., Silcox et al. 2009b, 2010a; Bertrand et al. 2019a) reached a divergent conclusion, with primates generally (including plesiadapiforms) having relative brain sizes that are typically a bit higher than found in other “archaic” groups. There are many ways those analyses could be improved. In particular, they are still heavily dependent on Radinsky’s (1978) endocranial volume estimates, which were calculated using double integration. As the database of virtual endocasts expands, it would be preferable to use a sample of endocranial volume estimates that are not so model dependent. Second, it would be beneficial to incorporate a phylogenetic factor (alongside a temporal one) into the analysis, rather than treating all non-primates as an undifferentiated mass (see discussion in Sect. 12.4.4).

The approach of formulating comparisons to other “archaic” mammals still does not answer the question of whether or not the earliest primates had larger (or differently organized) brains compared to their ancestors. The Radinsky (1978) sample that is central to such analyses is made up of carnivores and ungulates, which are only distantly related to Primates. As such, this sample does not provide the appropriate context to consider this question. With the expanded sample of closer primate relatives (i.e. rodents, lagomorphs, and apatemyids) available, it is starting to be possible to address this question (see Sect. 12.4.2).

12.3 Overview of General and Comparative Anatomy

12.3.1 Characterization of Cranial Endocast Morphology

12.3.1.1 Plesiadapiformes

There are reasonably complete endocasts published for four species of plesiadapiforms, in three famlies: Paromomyidae (Ignacius graybullianus: USNM 421608, Silcox et al. 2009b; UF 26000, Boyer et al. 2011; Long et al. 2015); Microsyopidae (Microsyops annectens: UW 12362, UW 14559, Silcox et al. 2010a); and Plesiadapidae (Plesiadapis cookei: UM87990, Gingerich and Gunnell 2005; Orliac et al. 2014; Plesiadapis tricuspidens: MNHN CR 125, Orliac et al. 2014; Kristjanson et al. 2016) (Fig. 12.2). Endocasts for two other species have been mentioned in abstracts, but have not yet been published in detail (Niptomomys cf. N. doreenae: USNM 530198, White et al. 2016; Carpolestes simpsoni: USNM 482354; Silcox et al. 2017b); discussion of these specimens here is limited to what was included in the abstracts. As noted above, Szalay (1969) published a partial latex endocast for the microsyopid Megadelphus lundeliusi (AMNH 55284; see also Radinsky 1977). He did not provide any quantitative data. One of us (MTS) located the remnants of the endocast in the AMNH collection, but unfortunately it is degraded beyond usefulness. Szalay (1969), Silcox et al. (2010a), and Chester et al. (2019) also provided some endocranial details from partial cranial specimens of Microsyops annectens (AMNH 12595), Microsyops sp. cf. M. elegans (UM 99843) and Torrejonia wilsoni (NMMNH P-54500) respectively.

All the plesiadapiform endocasts that have been published show some basic points of similarity (Fig. 12.2). All have pedunculated olfactory bulbs separated from the rostral end of the cerebrum by a well demarcated circular fissure (Fig. 12.4a, b) implying that there was no overlap of the cerebrum onto the olfactory bulbs. The volume of the olfactory bulbs relative to the endocast as a whole is around 5% (Table 12.1) for P. tricuspidens, M. annectens, and I. graybullianus. Orliac et al. (2014) produced a partial virtual endocast of P. cookei, which yielded a somewhat higher (7.8%) estimate of relative olfactory bulb size. However, this value is likely inflated as much of the ventral aspect of the endocast caudal to the olfactory bulbs is missing (see Orliac et al. 2014: fig. S2). In contrast, White et al. (2016) found that the olfactory bulbs in Niptomomys cf. N. doreenae were relatively somewhat larger (8.61%); in this case the estimate (made from the more complete side of the endocast) likely represents a real difference from the other plesiadapiforms. Whether this large size is interpreted as primitive or derived depends on the taxa used for comparison. Early rodents (i.e. Paramys copei, 6.05%; Paramys delicatus, 4.75%; Bertrand et al. 2016; Fig. 12.5b) and an early lagomorph (Megalagus turgidus 3.96%; López-Torres et al. 2020; Fig. 12.5a) have olfactory bulbs that are smaller than reconstructed for Niptomomys cf. N. doreenae, which suggests that Niptomomys may have been specialized rather than exhibiting the primitive condition. However, the large olfactory bulbs of the basal apatemyid Labidolemur kayi (~12–15%; Silcox et al. 2011; Fig. 12.5c) send a contrary message.

Fig. 12.4
figure 4

Virtual endocast of Ignacius graybullianus (USNM 421608) in (a) left lateral, (b) dorsal and (c) ventral views, labelled with key structures discussed in the text. Endocast originally published in Silcox et al. (2009b). Scale = 5 mm

Fig. 12.5
figure 5

Endocasts for early primates and members of closely related groups in dorsal view. (a) lagomorph Megalagus turgidus (FMNH UC 1642); (b) rodent Paramys delicatus (AMNH 12506); (c) apatemyid Labidolemur kayi (composite endocast based on USNM 530208 [purple] and USNM 530221 [teal]); (d) plesiadapiform Ignacius graybullianus (USNM 421608); (e) euprimate Adapis parisiensis (NHM M 1345). Endocasts originally published in Silcox et al. (2009b, 2011), Harrington et al. (2016), Bertrand et al. (2016), and López-Torres et al. (2020)

In terms of the cerebrum, all plesiadapiforms are similar in having a narrow rostral end (suggesting small frontal lobes) and a caudal extent that does not fully cover the midbrain (Figs. 12.2 and 12.4a, b). There is some variation in the degree of exposure of the colliculi: in I. graybullianus (Fig. 12.4b; Silcox et al. 2009b), Plesiadapis cookei (Gingerich and Gunnell 2005), P. tricuspidens (Orliac et al. 2014), and Carpolestes simpsoni (Silcox et al. 2017b) a pair of colliculi (presumably the caudal or inferior colliculi) are exposed. The inner surface of the cranium of the palaechthonid plesiadapiform (Torrejonia wilsoni; Chester et al. 2019: fig. 3) also shows indentations for exposed colliculi. Within Microsyopidae there is some variability. A pair of colliculi are exposed in Niptomomys cf. N. doreenae (White et al. 2016), Microsyops sp. cf. M. elegans (Silcox et al. 2010a), and one specimen of Microsyops annectens (UW 14559; Silcox et al. 2010a). However, in the other known specimen of M. annectens (UW 12362) and in Megadelphus lundeliusi (see Szalay 1969: pl. 41) the colliculi are not exposed; although there is a small patch of midbrain visible, and it appears as though the transverse sinus is roofing the midbrain rather than the cerebrum (Szalay 1969; Silcox et al. 2010a). This contrast may relate to some small expansion of the cerebrum within the Microsyopidae, perhaps associated with more visual processing (Silcox et al. 2010a), because the taxa in which the colliculi are not consistently exposed are later occurring. Edinger (1964) made the point that exposure of the midbrain on the endocast is not necessarily primitive—it could also result from expansion of the colliculi for functional reasons. Interestingly, newborn Tupaia actually exhibit exposed rostral (superior) colliculi (Tigges and Shantha 1969), which is likely a reflection of the fact that the relevant part of the brain is very expanded in treeshrews (Kaas 2002). With respect to plesiadapiforms, however, midbrain exposure seems likely to be primitive, based on comparison to a range of relevant outgroups. The colliculi are exposed in the apatemyid L. kayi (Silcox et al. 2011; Fig. 12.5c), and there is very broad midbrain exposure in Rhombomylus turpanensis (Meng et al. 2003: fig. 51). Among early rodents, all the ischyromyids show some degree of midbrain exposure, with a couple of species showing clearly exposed colliculi (Bertrand et al. 2019a: table S14). The endocast of Megalagus turgidus does not exhibit exposed colliculi but does have a small patch of exposed midbrain (López-Torres et al. 2020; Fig. 12.5a). In sum, then, it appears likely that the exposure of the midbrain is primitive for plesiadapiforms, and likely characterized the common ancestor of Euarchontoglires.

No plesiadapiforms known from adequate material possess a Sylvian sulcus, or a well-defined temporal pole, which means that the ventral aspect of the cerebrum is nearly in line with, or ventral to, the brain stem (Fig. 12.4a). The same is also true in L. kayi, R. turpanensis, M. turgidus, and in early rodents (Meng et al. 2003; Silcox et al. 2011; Bertrand et al. 2016, 2019a; López-Torres et al. 2020). Interestingly tupaiid treeshrews actually have fairly well-defined temporal poles (e.g. see Le Gros Clark 1924: fig. 1), and some modern sciurids also develop a similar morphology, with at least one species (Rhinosciurus laticaudatus) even exhibiting a Sylvian sulcus (Bertrand et al. 2017: fig. 5). The fact that the temporal lobe is relatively small in the most basal living treeshrew, Ptilocercus lowii (e.g. see Le Gros Clark 1926: fig. 17), and in the fossil sciurid Cedromus wilsoni (Bertrand et al. 2017) suggests that the primitive state for Euarchontoglires is likely to be a poorly defined temporal pole, and suggest that the superficial similarity between euprimate (see below) and treeshrew endocasts in this feature arose independently.

The larger plesiadapiforms (M. cf. elegans, M. lundeliusi, M. annectens, P. tricuspidens, P. cookei) all possess a lateral (=coronolateral, longitudinal, marginal) sulcus that runs approximately parallel to the superior sagittal sinus (Silcox et al. 2010a; Orliac et al. 2014; Fig. 12.2). The absence of this sulcus in the smaller plesiadapiforms (e.g., Ignacius graybullianus; Silcox et al. 2009b, 2010a; Fig. 12.4) likely relates to the fact that their endocranial volumes are less than 5 cc, the cut-off point below which brains typically fail to exhibit neocortical sulci (Macrini et al. 2007). There is some variability in the presence of the lateral sulcus in other fossil euarchontoglirans (Silcox et al. 2011; Bertrand et al. 2016, 2019a; López-Torres et al. 2020), but this likely reflects variation in size rather than being informative about primitive states. Similarly, modern dermopterans have a lateral sulcus (Gingerich and Gunnell 2005: fig. 5) but modern treeshrews do not (Le Gros Clark 1924, 1926), which is probably a matter of their differing cranial capacities.

The location of the rhinal sulcus (=fissure; ventral edge of the neocortex) has been interpreted as corresponding to the orbitotemporal canal (=sinus canal) in plesiadapiforms that preserve the relevant region (Silcox et al. 2009b, 2010a; Orliac et al. 2014); these features are associated in modern lemuriforms (Martin 1990) and at least some rodents (Bertrand and Silcox 2016; Bertrand et al. 2016, 2017, 2018, 2019a). The orbitotemporal canal is located approximately two-thirds of the way down the lateral side of the cerebrum in M. annectens; the position was likely similar in I. graybullianus (Fig. 12.4a; see also Long et al. 2015: fig. 3F) and possibly P. tricuspidens, although compression in the latter makes its position difficult to discern. As noted by Silcox et al. (2010a) and Orliac et al. (2014), the indentation identified as the rhinal fissure on the reconstructed endocast of P. cookei by Gingerich and Gunnell (2005) is likely to be too far ventral. Instead, P. cookei may have been like M. annectens, and possibly M. lundeliusi, in having an additional faint neocortical sulcus (?suprasylvian; Silcox et al. 2010a). The significance of the position of the rhinal sulcus is discussed further below (Sect. 12.4.4).

The morphology of the cerebellum in I. graybullianus and M. annectens is similar. In both cases there is a well demarcated vermis separated from the lateral lobes by paramedian fissures (Silcox et al. 2009b, 2010a; Figs. 12.2 and 12.4b). There is no clear evidence of a fissura prima. The petrosal lobules (often referred to as the paraflocculi) are well-rounded and connect to the rest of the cerebellum with a short stem. In both cases the cerebellum accounts for between a quarter and a third of the total length of the endocast. It is difficult to form more refined quantitative comparisons about the cerebellum, since it is challenging to separate it from other parts of the brain in endocasts. However, based on the relative length of the cerebellum, it could be interpreted as making up a smaller proportion of the brain in Plesiadapis than in other plesiadapiforms, because it only accounts for about 17% of the total length of the endocast in P. tricuspidens (Orliac et al. 2014). Damage to the relevant specimen makes the precise position of the front of the cerebellum a matter of interpretation, so it would be beneficial to be able to assess this in another specimen (unfortunately the full length of the endocast is not preserved for P. cookei). The petrosal lobules are also quite distinctive in shape in P. tricuspidens compared to I. graybullianus and M. annectens, being more elongate and cylindrical, and less globular (Fig. 12.2), a contrast Orliac et al. (2014: p. 3) argue is real based on the “perfect preservation of both petrosals” in P. tricuspidens.

12.3.1.2 Adapoids and Omomyoids

This discussion will focus on species for which three-dimensional endocasts are available (i.e., the adapoids Smilodectes gracilis, Adapis parisiensis, and Notharctus tenebrosus; the omomyoids Microchoerus erinaceus and Necrolemur antiquus; and Rooneyia viejaensis; Gazin 1965; Gingerich and Martin 1981; Kirk et al. 2014; Harrington et al. 2016, 2020; Ramdarshan and Orliac 2016) (Fig. 12.3) with additional details from specimens known only from natural endocasts that are partially visible through breaks in the cranium (see Gurche 1982: fig. 6) as warranted. As noted above there are endocranial volume estimates that have been calculated for the adapoids Pronycticebus gaudryi and Leptadapis magnus (Martin 1990), but these species are not yet known from published endocasts.

The adapoids and omomyoids known from endocasts are similar to plesiadapiforms in having pedunculated olfactory bulbs separated from the cerebrum by a distinct (if narrow) circular fissure (Figs. 12.3 and 12.5). The volume of the olfactory bulbs relative to the overall endocranial volume is typically lower in euprimates than in plesiadapiforms. For specimens with volumes directly measured from CT data the range of variation is 0.94% (Rooneyia viejaensis; Kirk et al. 2014) – 2.40% (Adapis parisiensis; Harrington et al. 2016). Estimates for taxa not yet known from virtual endocasts extend this range (i.e., 3.4% for Tetonius homunculus; Gurche 1982; Ramdarshan and Orliac 2016). These values generally lie within the range of variation observed for living strepsirrhines (0.39%–3.38%; Stephan et al. 1981; Kirk et al. 2014), but above the value for Tarsius sp. (0.53%; Stephan et al. 1981; Kirk et al. 2014). Although the contrast between plesiadapiforms and euprimates could be interpreted as evidence for reduced importance in the sense of smell through evolutionary time, it is worth noting that the distinction mostly disappears when the size of the olfactory bulbs is assessed against body mass rather than endocranial volume (i.e., see Harrington et al. 2016: fig. 12). Therefore, the difference in relative size may relate more to increases in other parts of the brain than to decreases in the size of the olfactory apparatus, a point Martin (1990) also made with respect to the relative size of the olfactory bulbs in living strepsirrhines compared to non-primates (see Martin 1990: fig. 8.16; see also Heritage 2014).

The presence of a clear circular fissure on the endocast is a contrast with the situation in living euprimates, in which the cerebrum typically overlaps at least somewhat onto the olfactory bulbs. Alongside the relatively narrow rostral end of the cerebrum evident in adapoids and omomyoids (Fig. 12.3), this lack of overlap could signal a lesser development of the frontal lobes in primitive euprimates relative to extant species (Radinsky 1970; Jerison 1973; Kirk et al. 2014), although actually quantifying the relative size of this part of the brain is not possible (Jerison 2007). In contrast to plesiadapiforms, however, the cerebrum has a well-defined temporal pole in all euprimates known from endocasts. Associated with this, most fossil euprimate taxa have a fairly well distinguished Sylvian sulcus, which is a trait that has long been considered a distinctive feature of the primate brain (Elliot Smith 1902; although as noted above, this feature does occasionally develop in other groups; Bertrand et al. 2017). The sole exception to this generality among fossil euprimates is Smilodectes gracilis, which is variable in the presence of the Sylvian sulcus (Gazin 1965; Harrington et al. 2016; it is also only weakly expressed in a specimen of N. tenebrosus, AMNH 127167). The importance of this variable presence is somewhat ambiguous because it could reflect obscuring by dural vessels or thick meningeal tissues rather than a real absence from the brain (see discussion in Harrington et al. 2016). In any case, the expansion of the cerebrum (so that in lateral view the temporal pole extends ventrally beyond the level of the ventral border of the brain stem; Fig. 12.3) is a distinct difference from plesiadapiforms (Figs. 12.2 and 12.4a), suggestive of expansions to the temporal lobe.

In all the fossil euprimates known from endocasts that preserve the relevant area, the orbitotemporal canal (and therefore presumably the rhinal fissure) is located near the ventral extent of the temporal lobe (Fig. 12.3), in a position that is farther ventral than observed in the plesiadapiforms that preserve this feature, and similar to some small-bodied modern strepsirrhines (e.g., Microcebus; Kirk et al. 2014: fig. 4). As discussed below, this contrast is likely associated with a relative expansion of the neocortex at the euprimate node. Expansion of the cerebrum distally is also likely associated with increased neocorticalization, so that there is no exposure of the midbrain on the surface of the endocast (Fig. 12.3), unlike in plesiadapiforms (Fig. 12.2). This contrast suggests, therefore, some expansion of the occipital lobe with the evolution of Euprimates.

As in the plesiadapiforms, the larger taxa (Adapis parisiensis, Smilodectes gracilis, Notharctus tenebrosus; Gazin 1965; Gingerich and Martin 1981; Gurche 1982; Harrington et al. 2016) among the adapoids and omomyoids have a well-defined lateral sulcus running approximately parallel to the superior sagittal sinus, but this feature is missing from the smaller forms (Rooneyia viejaensis, Tetonius homunculus, Necrolemur antiquus Radinsky 1970; Kirk et al. 2014; Ramdarshan and Orliac 2016; Harrington et al. 2020; Fig. 12.3). A lateral sulcus has been identified in Microchoerus erinaceus (Ramdarshan and Orliac 2016; Fig. 12.3), which is somewhat surprising because that species’ endocranial volume is 4.26 cc, and so below the 5 cc boundary that is typically associated with lissencephaly (Macrini et al. 2007). Endocasts of S. gracilis and N. tenebrosus are variable in the expression of a faint dorsolateral sulcus in the region between the lateral sulcus and the orbitotemporal canal (e.g., see Harrington et al. 2016: fig. 5F), which has been referred to as a possible suprasylvian sulcus (Gurche 1982; Harrington et al. 2016); the position is similar to the faint ?suprasylvian sulcus evident in the plesiadapiform M. annectens (and also possibly M. lundeliusi and P. cookei; Silcox et al. 2010a). The expression of this feature varies not only among specimens, but even within particular specimens (e.g., it is better defined on the left size of AMNH 127167 [N. tenebrosus] than it is on the right; see Harrington et al. 2016: fig. 5). A shallow sulcus near the anteroventral border of the temporal lobe was identified in Microchoerus erinaceus (i.e., “temporal sulcus” of Ramdarshan and Orliac 2016: fig. 3C). Interestingly, a faint sulcus in a very similar position was identified in two specimens (Montauban 9 [MaPhQ 289] and BMM 4490) of N. antiquus by Gurche (1982; fig. 6f, g); he likened it to the postsylvian sulcus of Tarsius, which would be interesting in light of the historical tie suggested between those taxa (Rosenberger 1985). However, this feature is not evident on the virtual endocast of Montauban 9 (Harrington et al. 2020).

In general, it would be fair to say that early euprimate brains are characterized by the usual presence of the Sylvian sulcus, with evidence of independent development of additional subtle sulci, starting with the longitudinal sulcus, as brains start to increase in size. The pair of sulci on the relatively small brain of M. erinaceus stands out as notable, although it is unclear if this pattern represents a part of any kind of larger evolutionary picture.

All early euprimates known from endocasts (Fig. 12.3) share a basically similar morphology of the cerebellum with the plesiadapiforms Ignacius graybullianus and Microsyops annectens (Figs. 12.2 and 12.4). There is a clear division, by way of paramedian fissures, between the vermis and the lateral lobes, and the petrosal lobule is globular and attached to the rest of the cerebellum by a short stem. It is difficult to formulate any quantitative comparisons about the cerebellum from the endocranial evidence, because in early euprimates there are varying degrees of coverage of this part of the brain by the cerebrum (not covered in S. gracilis, N. tenebrosus, A. parisiensis; partly covered in N. antiquus, T. homunculus, N. antiquus, M. erinaceus, R. viejaensis; Harrington et al. 2016, 2020; Fig. 12.3). The relative length of the cerebellum on the ventral surface of the endocast likely has more to do with the degree of flexion of the cranium than with the actual size of the cerebellum. So, for example, the cerebellum appears very short in dorsal view in R. viejaensis (Kirk et al. 2014: fig. 3A) and much longer in A. parisiensis (Harrington et al. 2016: fig. 9B), but this is likely because the cranium of R. viejaensis is much more strongly flexed (with a cranial base angle of 176° compared to 187° in A. parisiensis; Harrington et al. 2020: table 1). Gurche (1982) provided an equation for calculating relative cerebellar size, but because it is based on brain mass, it is not possible to use it to consider cerebellar size as independent from overall brain size. For this reason, unfortunately data from the endocasts of early primates cannot currently contribute to debates about the relative importance of the cerebellum in primate evolutionary history (e.g. Barton 2012).

12.3.2 Spaces Associated with Cranial Blood Supply

Endocasts of early primates possess casts of several spaces associated with arterial blood supply and venous drainage of the brain and cranium. Generally, the brains of fossil and extant haplorhine primates (including omomyoids), fossil anthropoids, most adapoids, and some plesiadapiforms (e.g., Microsyops annectens) are thought to be supplied by the vertebral artery and the promontorial branch of the internal carotid artery, whereas extant strepsirrhines, subfossil lemurs, some adapoids (e.g., Adapis parisiensis) and some plesiadapiforms (e.g., Ignacius graybullianus) are believed to have had non-patent (i.e., non-functional and/or absent) promontorial arteries (Bugge 1974, Conroy and Wible 1978; MacPhee and Cartmill 1986, Boyer et al. 2016). Among extant strepsirrhines with non-patent promontorial arteries, several groups (e.g., cheirogaleids and lorisiforms) supplement their encephalic blood supply via branches of the ascending pharyngeal artery, which stems from the external carotid arteries (Cartmill 1975; MacPhee and Cartmill 1986). There is some ambiguity in the pattern of evolution of internal carotid arterial reduction in strepsirrhine evolution, driven in part by variation among adapoids (e.g., the promontorial artery was involuted in Adapis parisiensis but not in its close relative Leptadapis), which indicates that there must have been some measure of homoplasy in this trait (Boyer et al. 2016).

The impressions of grooves, which presumably marked the paths of the promontorial arteries, are observed caudal or lateral to the cast of the hypophyseal fossa on the ventral surface of the endocasts of several species. These species include the plesiadapiform Microsyops annectens (Silcox et al. 2010a), the omomyoid Necrolemur antiquus (Harrington et al. 2020) and the adapoids Notharctus tenebrosus (Harrington et al. 2016) and Smilodectes gracilis (Gazin 1965; Harrington et al. 2016). These species are consistent with those identified by Boyer et al. (2016) to have likely had patent promontorial arteries (i.e., that supplied the brain), on the basis of the area of the ossified promontorial canal relative to brain size.

Inferring arterial blood supply to the brain from endocasts is limited in species which do not have patent promontorial arteries. The vertebral arteries enter the endocranial space through the foramen magnum and do not leave a cast of their course on endocasts. In addition, branches of the ascending pharyngeal arteries supplying the brain enter the endocranium via a foramen lacerum medium (Cartmill 1975; Conroy and Packer 1981; MacPhee and Cartmill 1986), which may also pass other structures and thus may not be correlated to the presence of the artery.

Far more numerous than the traces of arterial features on the endocast are the impressions of venous features. Chief among these are venous sinus spaces enclosed by folds in the dura mater. In mammals, the superior sagittal sinus, which forms at the apex of the falx cerebri, drains into the transverse sinus (Fig. 12.4b; sometimes referred to as the lateral sinus, e.g., Gazin 1965 and Gingerich and Martin 1981) in the edge of tentorum cerebelli before continuing to the sigmoid sinus. In turn, the sigmoid sinus (Fig. 12.4b), as well as the inferior petrosal sinus on the ventral surface of the brain, empties into the internal jugular vein in the jugular foramen, which is one major path for blood exiting the endocranial cavity (Butler 1967; Wible 1990). Primitively for eutherian mammals, the transverse sinus is also continuous with a sinus variably called the petrosquamous or capsuloparietal emissary vein, which drains into the postglenoid vein exiting the endocranial cavity via the postglenoid foramen (Wible 1990; Wible and Zeller 1994). In treeshrews, the capsuloparietal emissary vein is also continuous anteriorly with the cranio-orbital sinus, which travels along the cranio-orbital canal to the orbits (Wible 2011; Wible and Zeller 1994). Hence, the capsuloemissary vein, cranio-orbital sinus, and postglenoid vein share a confluence in treeshrews. With a few exceptions, endocasts of plesiadapiforms and early euprimates preserve features which suggest they shared the above-described general primitive pattern of endocranial venous drainage (Fig. 12.4).

The impression of the superior sagittal sinus is prominent on the dorsal surface of many early fossil primate endocasts, particularly on the surface of the caudal half of the cerebrum (Fig. 12.4b). Macrini et al. (2007) suggested that the absence of a cast of the superior sagittal sinus may indicate a relatively deep position of this sinus within the meninges in life. This could suggest that in certain endocasts where the superior sagittal sinus is more prominent caudally (e.g., as seen in adapoids; Harrington et al. 2016; Fig. 12.3), that the sinus was deeper within the meninges surrounding the rostral half of the brain, and/or perhaps became more salient as it collected blood from more contributing veins caudally.

The cast of the confluence of the superior sagittal sinus and transverse sinuses are also well-preserved on the dorsal surface of early primate endocasts (Fig. 12.4b). The sigmoid sinus typically courses caudal to the petrosal lobules (Fig. 12.4b) but were either absent or not well-preserved on the endocasts of P. tricuspidens, M. erinaceus, and N. antiquus (Orliac et al. 2014; Ramdarshan and Orliac 2016; Harrington et al. 2020). On the ventral surface, bilateral casts of the inferior petrosal sinus have been identified on endocasts of early primates with the exception of P. tricuspidens and I. graybullianus (Orliac et al. 2014; Silcox et al. 2009b).

The portion of the petrosquamous sinus/capsuloparietal emissary vein connecting the transverse sinus to the postglenoid foramen (Fig. 12.4c) is evidently completely enclosed by bone in many plesiadapiforms, adapoids, and omomyoids, although this condition was not observed on the virtual endocast of P. tricuspidens or N. antiquus (Harrington et al. 2016, 2020; Orliac et al. 2014; Ramdarshan and Orliac 2016; Silcox et al. 2009b, 2010a). A distinct cast of the canal for the postglenoid vein and the orbitotemporal canal are also visible on virtual endocasts of early primates, except in that of N. antiquus (MaPhQ 289), for which it could not be discerned from a CT scan whether a definitive orbitotemporal canal was present (Harrington et al. 2020). It does not seem likely that this canal was entirely absent in Necrolemur, as M. erinaceus, which does possess a bilateral cast of the orbitotemporal canals on its endocast, and has been hypothesized to be a direct descendent of N. antiquus (Minwer-Barakat et al. 2017). Thus, it is unlikely that this primitive endocranial feature was lost in Necrolemur, then regained in Microchoerus; the more likely alternative is that its absence on the endocast is a product of preservation.

12.4 Brain Evolution and Paleobiological Inferences Based on Endocast Morphology

12.4.1 Morphological Brain Diversity

As detailed above, we now have some understanding of the form of the brain both in stem primates, and in early euprimates, and can reach some tentative conclusions about directions in evolutionary change occurring near the base of the primate tree. Plesiadapiforms can be inferred to have had quite primitive looking brains, sharing fundamental similarities with endocasts that have been reconstructed for early fossil rodents (i.e., ischyromyids; Bertrand and Silcox 2016; Bertrand et al. 2016, 2019a) and for a stem lagomorph (López-Torres et al. 2020). In particular, like the endocasts in those taxa, they have fairly large, pedunculated olfactory bulbs, have a cerebrum that does not overlap onto the circular fissure or entirely cover the midbrain, and lack a Sylvian fissure and a clearly demarcated temporal pole (Fig. 12.5). As in early rodents and Megalagus, larger plesiadapiforms develop a lateral sulcus, with their brains otherwise being basically lissencephalic (with the exception of the very shallow ?suprasylvian sulcus of M. annectens and possibly M. lundeliusi and P. cookei). As noted above, there is some ambiguity in the direction of evolutionary change in the relative size of the olfactory bulbs based on the conflicting signal from rodents and lagomorphs on one hand, and the apatemyid Labidolemur kayi on the other. So perhaps the basal primate node was associated with some decrease in the relative size of these bulbs (but perhaps not; see also Heritage 2014). In all, there are few clear indications of special similarities in the brain between plesiadapiforms and euprimates. One possible exception to this was highlighted by Orliac et al. (2014: p. 1), who suggested that, in spite of being at the low end of the known variation in plesiadapiforms for both EQ and relative neocortical size, P. tricuspidens was similar to euprimates in having a “…domed neocortex and downwardly shifted olfactory-bulb axis”, differing in this way from Ignacius graybullianus and Microsyops annectens. Phylogenetic analyses (e.g. Bloch et al. 2007; Silcox et al. 2010b; Chester et al. 2019) suggest that plesiadapids are more closely related to euprimates than paromomyids and microsyopids are. This shift could represent some re-organization of the brain in stem primates, prior to any kind of significant expansion in the relative size of the brain overall, or of the neocortex specifically. However, that conclusion is based on a very heavily pancaked specimen, so this inference merits testing in other plesiadapoid specimens (i.e., including carpolestids or saxonellids).

What is more certain is that there was a quite significant re-organization of the brain associated with the euprimate node, with all euprimates showing evidence of expansion in the temporal and occipital lobes (associated with the development of a Sylvian sulcus and strong temporal pole, and coverage of the midbrain) compared to plesiadapiforms. The more ventral position of the rhinal fissure suggests expansion of the neocortex (see Sect. 12.4.4). The relative size of the olfactory bulbs is lower, but this may represent stasis, where in other regions were expanding, rather than an actual decrease in their absolute size. In all, early euprimates have brains that are similar in morphology in many ways to extant small strepsirrhines, differing predominantly in an inferred lesser development of the frontal lobes. Whether this reorganization was associated with a significant increase in overall size is a matter of some debate (Harrington et al. 2016; Gilbert and Jungers 2017; see discussion above and in Sect. 12.4.2), but as noted above, if an increase did occur, it did not lead to relative brain sizes that were comparable to living primates in most cases.

12.4.2 Brain-Size Evolution and Encephalization Quotient

As detailed above, the availability of quantitative data on encephalization for both plesiadapiforms and early euprimates has increased significantly in the last 15 years (Gingerich and Gunnell 2005; Silcox et al. 2009b, 2010a; Kirk et al. 2014; Orliac et al. 2014; Harrington et al. 2016, 2020; Ramdarshan and Orliac 2016; Table 12.1). This information allows us to explore quantitatively the question of when increases in encephalization occurred in early primate evolution, placing this question within the updated evolutionary framework of Euarchontoglires. Quantitative encephalization data for fossil primates is also extensive for higher nodes of the tree, including anthropoids (Martin 1993; Begun and Kordos 2004; Bush et al. 2004a,b; Holloway et al. 2004; Guy et al. 2005; Nargolwalla et al. 2005; Falk 2007; Harvati and Frost 2007; Simons et al. 2007; Weston and Lister 2009; White et al. 2009; Kay et al. 2012; Gonzales et al. 2015; Ni et al. 2019) and crown strepsirrhines (Ryan et al. 2008). Therefore, the encephalization data collected from fossil primates, combined with the brain and body mass data that exist for a great diversity of living primates (Table 12.S1), allows us to comprehensively probe this question through the means of ancestral state reconstruction analyses. Taxa for which endocranial volume estimates were made from external measurements of the cranium were generally excluded from this analysis.

To accurately reconstruct deep nodes in the primate tree, such as those of the ancestral euprimate or the ancestral primate, it is necessary to include the same type of quantitative information for other euarchontoglirans. Previous attempts at reconstructing the ancestral euprimate relative brain size (Montgomery et al. 2010; Steiper and Seiffert 2012) used a sample exclusively made up of primates without putting them in an euarchontogliran context. Boddy et al. (2012) reconstructed the ancestral euprimate EQ using a mammalian tree that included Scandentia, Rodentia and Lagomorpha (but not Dermoptera), but that did not include fossils. Fortunately, recent work in the last decade has provided relevant data for dermopterans (San Martin-Flores et al. 2019), scandentians (San Martin-Flores et al. 2018), fossil rodents (Bertrand and Silcox 2016; Bertrand et al. 2016, 2017, 2018, 2019a; Ferreira et al. 2020), fossil lagomorphs (López-Torres et al. 2020), and apatemyids (Silcox et al. 2011), allowing for an examination of change in brain size on the primate (and euarchontogliran) tree that at least partially overcomes the limitations of previous studies.

The taxa for which there are estimates of endocranial volume and body mass available (Table 12.S1) were assembled into a supertree based on Kobayashi (1995), Takai et al. (2008), Silcox et al. (2010b), Roberts et al. (2011), Springer et al. (2012), Gudde et al. (2013), Baab et al. (2014), Martins Jr. et al. (2015), Strait et al. (2015), Ni et al. (2016, 2019), Byrne et al. (2018), Mongle et al. (2019), and Bertrand et al. (2021). This tree was used as the basis for an analysis of ancestral states for EQ in Mesquite 3.2 (Maddison and Maddison 2017) under parsimony (i.e., using the Analysis:Tree Trace All Characters Parsimony Ancestral States option). We performed the analysis using estimates of EQ based on both Jerison’s (1973) and Eisenberg’s (1981) equations, and for topologies that support both Sundatheria (i.e., treeshrews and colugos as sister taxa) and Primatomorpha (i.e., primates and colugos as sister taxa). Figures 12.6 and 12.S1 were made with the software FigTree and depict the results of the analysis using Jerison’s (1973) equation and the topology that supports Primatomorpha, while Table 12.2 includes reconstructed values for key nodes from all 4 analyses.Our results suggest that there is a marked increase in EQ from the ancestral primate to the ancestral euprimate nodes (Table 12.2, Fig. 12.6). Using Jerison’s (1973) EQ, the ancestral primate would have had an EQ of 0.41 and the ancestral euprimate an EQ of 0.68; using Eisenberg’s (1981) EQ, they would have had EQs of 0.57 and 0.92, respectively. These results are obtained using a phylogeny of Euarchontoglires that supports Primatomorpha (i.e., a monophyletic clade that includes Primates and Dermoptera; Janečka et al. 2007). There is a negligible change in these numbers if we use instead a phylogeny that supports Sundatheria; the ancestral primate node decreases its reconstructed EQs by only 0.01 (Table 12.2). Given these results, the ancestral primate is inferred to have been similarly encephalized to plesiadapiforms, dermopterans, ischyromyid rodents, and apatemyids, but also to adapoids. The ancestral euprimate would have had a higher EQ, more similar to those of omomyoids. There are additional increases associated with the lineages leading to Strepsirrhini and Haplorhini, and further increases within those clades, highlighting the rampant parallelism that was clearly a characteristic of the evolution of brain size in Primates (Table 12.2; see discussion below).

Fig. 12.6
figure 6

Visualization of the ancestral state reconstruction analysis on a supertree representing a hypothesis of relationships among Euarchontoglires (see Sect. 12.4.2). Colors represent values of Jerison’s (1973) encephalization quotient (EQ), with colder colours showing lower EQ values and warmer colours showing higher EQ values: 0.1–0.5, purple; 0.5–0.9, dark blue; 0.9–1.3, medium blue; 1.3–1.7, light blue; 1.7–2.1, dark green; 2.1–2.5, light green, 2.5–2.9, yellow; 2.9–3.5, orange; 3.5–3.9, light red; over 3.9, dark red. Fossils marked with a red dot. The analysis was performed in Mesquite 3.2 (Maddison and Maddison 2017) using parsimony. Combined cladogram from Kobayashi (1995), Takai et al. (2008), Silcox et al. (2010b), Roberts et al. (2011), Springer et al. (2012), Gudde et al. (2013), Baab et al. (2014), Martins Jr. et al. (2015), Strait et al. (2015), Ni et al. (2016, 2019), Byrne et al. (2018), Mongle et al. (2019), and Bertrand et al. (2021). The current tree supports Primatomorpha. For a more detailed tree, see Fig. 12.S1. Node names and associated ancestral state reconstruction values are given in Table 12.2

Table 12.2 Encephalization quotients for critical nodes reconstructed based on the analysis detailed in Sect. 12.4.2 and figured in Figs. 12.6 and 12.S1

Although one interpretation of this pattern could be that the strepsirrhine-like brain organization that is observed in early euprimates (Kirk et al. 2014; Harrington et al. 2016, 2020; Ramdarshan and Orliac 2016) was associated with a notable increase in encephalization, the fact that all the adapoids in our sample (Table 12.S1) have EQs that are notably below the value inferred for the ancestral euprimate complicates this interpretation. As it stands, in our analysis a reversal to a lower EQ is reconstructed as having occurred in adapoids. This is one possibility, but it is also worth considering whether or not this pattern is a product of the ancestral state reconstruction methodology, and of this particular topology. Specifically, the location of the middle Eocene Rooneyia (with an EQ in the range of living strepsirrhines) at the base of the tarsiiform clade in this topology is driving up the reconstructed primitive euprimate value. It is questionable whether the endocast of Rooneyia viejaensis is a good representative of what is primitive for that clade, in light of its specialized morphology and the late age of this species (Rosenberger et al. 2008; Kirk et al. 2014). These ambiguities mean that the differing interpretations of Harrington et al. (2016) and Gilbert and Jungers (2017) about whether shape changes preceded size increases in the earliest phases of euprimate evolution remain in contention. Endocranial data for more basal members of the tarsiiform clade (i.e., older and/or more primitive omomyoids) would likely help to resolve this issue.

The reconstructed EQ value for the ancestral euarchontan is actually higher than that of the ancestral primate, but lower than that of the ancestral euprimate (Table 12.2). Whereas it is possible that the primate lineage suffered a decrease in EQ at its basalmost node, it is important to acknowledge that the most closely related taxa (Dermoptera and Scandentia) are solely composed of extant species, since no fossil colugo or treeshrew crania have been recovered. The offset between the estimates for the ancestral euarchontan and the ancestral euarchontogliran (Table 12.2) may also reflect this issue, since all of the included members of Glires are fossil taxa. Extant treeshrews are particularly encephalized and they certainly have an important impact on the reconstruction of the basal euarchontan node. However, it can be concluded that there is no clear evidence for an increase in relative brain size at the basal primate node; as such, our analysis supports Radinsky’s perspective that the most ancient primates were not necessarily encephalized over their mammalian contemporaries in the historical debate (see Sect. 12.2).

Extant strepsirrhines show the lowest EQ values among modern primates. Here we have considered adapoids as stem strepsirrhines; as noted above they have very low EQs (particularly Notharctus and Adapis, Harrington et al. 2016). This probably explains the low EQ inferred for the ancestral strepsirrhine. Among living strepsirrhines, there are a few reversals in EQ that stand out. Lepilemurids seem to have particularly low EQs compared to other lemuriforms. Lepilemurids are highly folivorous but have also been observed to practice caecotrophy (i.e., the reingestion of soft faeces or caecotrophs, Hladik 1978), which serves to improve the absorption of vitamins and microbial proteins (Hirakawa 2001). It is possible that the suboptimal absorption of nutrients from plant material in lepilemurids serves as a limiting factor in brain development. Another reversal among lemuriforms pertains to Cheirogaleus. This might be explained by strong seasonal variation in body mass in dwarf lemurs. Cheirogaleus is unusual among primates in storing large amounts of fat subcutaneously during the rainy season to prepare for a long period of torpor during the winter months, which makes their body mass increase up to 50% (Lemelin and Schmitt 2004). However, the sources we used for Cheirogaleus’ body mass (Stephan et al. 1981; Boddy et al. 2012) do not report what time of the year they were taken, so it is hard to tell if this is the true reason behind the low EQ in this genus. The high degree of variation in body mass throughout the year will nonetheless impact the calculation of the EQ in that genus, with Cheirogaleus having its highest EQ after finishing torpor and its lowest before starting it, which is a good example of why EQ is a problematic tool to measure intelligence.

There is a consistent association in our analysis between lower EQ values and folivory (see also DeCasien et al. 2017). A prime example is the clear dichotomy in EQ trends between the cercopithecine and the colobine radiations (Table 12.2). There are a couple of explanations, not necessarily mutually exclusive, for this pattern. The Expensive-Tissue Hypothesis (Aiello and Wheeler 1995) suggests that the metabolic requirements of relatively large brains are offset by a corresponding reduction of the gut. Colobines, which are largely folivorous cercopithecoids, have stomachs that differ from any other primate and resemble those of ungulates, with a pseudoruminant anterior fermentation area in a large multichambered stomach (Fleagle 2013). Another possible explanation is that folivores depend on food that is more easily accessible and more predictable in time and space than that of frugivores. Consequently, folivores may not experience the types of cognitive demands for efficient exploitation of their food supply encountered by primates in other dietary categories (Clutton-Brock and Harvey 1980). This pattern is also observed in other areas of the primate tree: gorillas compared to other great apes, alouattines compared to atelines, Avahi compared to other indriids, and lepilemurids compared to other lemuriforms.

Finally, there are a few lineages that show evidence of increased EQ that are worth mentioning. The hominin lineage, of course, stands out for clustering the highest EQs in the tree. Other groups with high EQ compared to their close relatives are cebines and aye-ayes. Cebinae groups together some of the most encephalized platyrrhines, which may have some relationship to the use of tools by cebines for a variety of purposes. For example, they are known to use stones to crack nuts, sticks to strike a conspecific or push objects, or leaves to be used as a cup, making them more proficient in tool use than most other non-ape anthropoids (Visalberghi 1990; Phillips 1998). The aye-aye (Daubentonia madagascariensis) is one of the most encephalized strepsirrhines. Aye-ayes evolved a context-specific form of manual extractive foraging involving a long, thin third digit for extracting grubs from within tree bark. This type of convergent evolution with other primates who practice omnivorous extractive foraging (i.e., cebines, chimpanzees and humans) may potentially be related to the observed parallel increase in encephalization in these lineages (Gibson 1986; Kaufman et al. 2006; Parker 2015). However, aye-ayes do not achieve the same level of sensorimotor cognition and comprehension of tool use as their anthropoid relatives do (Sterling and Povinelli 1999).

12.4.3 Sensory Evolution: Vestibular Sense, Vision, Hearing, Olfaction, Taste, etc.

As the brain is where sensory input is processed into actionable information, the evolution of the primate brain from a sensory perspective has become the subject of extensive research. Exploration into the connection between sensory adaptation and brain evolution operates on Jerison’s (1973) Principle of Proper Mass, which ties the size of a brain structure to the information processing requirements of its function. This principle therefore suggests that an adaptation requiring an increase in the information sent to certain neural tissues will result in an increase in the size of those tissues. This principle serves as the foundation for interpreting size changes in the brain overall, and in specific neuroanatomical structures. Whereas much research into the sensory specialization of the primate brain has focused on smaller, more functionally specific brain regions (e.g. the striate cortex and the parvocellular and magnocellular layers of the lateral geniculate nucleus; Barton 1998), comparable analyses are largely not possible in endocast analyses as endocasts cannot provide information about internal structures. Consequently, only structures which can be measured accurately on the surface of the endocast are discussed here. Traditionally, these brain regions have included the neocortex, responsible for processing visual, auditory, somatosensory, motor, sensorimotor, and prefrontal sensory information (Kaas 2012); and the olfactory bulbs, responsible for processing olfactory information (Heritage 2014).

As one of the defining features of primate sensory adaptation (Silcox et al. 2007), specializations of the visual system have been thoroughly examined in the primate brain. Often, visual specializations are cited as a driving force behind primate encephalization and the expansion of the neocortex (Barton 1996, 1998; Kirk 2006) as a large portion of the neocortex is devoted to processing visual information, most notably in diurnal anthropoids (Felleman and Van Essen 1991; van Essen et al. 1992). To this end, several analyses have focused on the scaling relationship between visually demanding ecological behaviors and the size of the neocortex within extant haplorhines and strepsirrhines (Barton 1996, 1998). Overall, these analyses indicate that haplorhines have significantly larger neocortices relative to the size of the rest of the brain, compared to strepsirrhines, and neocortex size is correlated with ecological behaviors including social group size, diet, and activity pattern (Barton 1996; DeCasien et al. 2017; DeCasien and Higham 2019). Among extant primates, diurnal frugivorous anthropoids living in large groups exhibit the highest degree of cortical expansion (Barton 1996; DeCasien and Higham 2019). It has been suggested that this scaling relationship is the product of the increased visual demands of primate communication and/or visually oriented foraging behaviors (Barton 1996, 1998, 2000).

However, the neocortex is also responsible for functions outside of vision (Joffe and Dunbar 1997). Analyses into more functionally specific visual structures, including the striate cortex and the lateral geniculate nucleus, identify similar scaling relationships associated with activity pattern, diet, and social group size (Barton 1998; DeCasien and Higham 2019). But, only a small portion of the variation in total neocortex size can be attributed to expansion of these visual structures (lateral geniculate nucleus: r2 = ~ 0.18; p = 0.014 and striate cortex: r2 = ~ 0.14; p = 0.03, Barton 1998). Given the diversity of sensory functions the neocortex performs, it is somewhat problematic to use neocortical expansion as an indicator for specialization in a single sensory modality. Nevertheless, researchers have examined the size and shape of the neocortex in connection with other ecological factors to help explain variation between closely related fossils. For example, caudal expansion of the neocortex, where the striate visual cortex is located, in later occurring microsyopids compared to other stem primates may indicate greater visual specialization among these taxa (Silcox et al. 2010a). Similarly, the lack of midbrain exposure in early fossil euprimates may be related to expansion of the neocortex related to improvements to visual processing (Harrington et al. 2016).

Brain size has also been examined in relation to the total amount of visual input. Kirk (2006) examined the relationship between total endocranial volume and optic foramen area, as the latter is strongly correlated with the size of the optic nerve and the number of ganglion cells in the retina (Kay and Kirk 2000; Kirk and Kay 2004). Body mass-controlled analysis of the relationship between these two variables in a large sample of extant primates indicated that visual input is significantly correlated with brain size, as relative orbital foramen area accounts for 43% (p < 0.0001) of the variation found in relative endocranial volume (Kirk 2006). Furthermore, anthropoids were found to have relatively larger optic foramina, indicative of increased visual input, and correspondingly larger brains compared to strepsirrhines regardless of ecology. The same analysis was performed on six fossil euprimates: three late Eocene adapoids Adapis parisiensis, Leptadapis sp., Pronycticebus gaudryi; a late Eocene omomyoid, Necrolemur antiquus; the early Oligocene stem anthropoid, Parapithecus (=Simonsius) grangeri; and Rooneyia viejaensis (see discussions above about its taxonomic position). The three adapoids, A. parisiensis, Leptadapis sp., and P. gaudryi, fell outside of the extant primate distribution, having relatively small orbital foramen areas associated with relatively small endocranial volumes (Kirk 2006). The haplorhines, N. antiquus and P. grangeri, along with R. viejaensis, plot within the distribution of extant primates (Kirk 2006). These results were interpreted to reflect a grade-shift in brain size between haplorhines and strepsirrhines that was linked to the amount of visual input to the brain. This point may relate to the ambiguities discussed above about when EQ increased on primate evolution (see Sect. 12.4.2)—specifically the increase inferred as pertaining to the primitive euprimate may be a primarily haplorhine event. These results are also consistent with other research which indicates that haplorhines, specifically anthropoids, are visually specialized as they possess greater degrees of orbital convergence (Ross 1995), greater visual acuity (Kirk and Kay 2004), and in some cases, trichromatic vision (Regan et al. 2001). However, it is worth noting that this conclusion depends in part on an estimate of endocranial capacity in N. antiquus that has since been re-assessed (Harrington et al. 2020).

Extant primates have long been considered to have a poor sense of smell (microsmatic), an idea that can be traced back to Elliot-Smith (1927) who suggested that olfaction would have been less important to primates than to other mammals because of their arboreal niche. Whether or not extant primates are microsmatic has been and continues to be discussed from genetic, behavioral, and anatomical perspectives (Smith et al. 2007). Concerning neuroanatomy, numerous studies have identified a clear grade-shift in the size of the olfactory bulbs relative to brain size between haplorhines and strepsirrhines, with haplorhines having significantly smaller olfactory bulbs (Stephan et al. 1981; Baron et al. 1983; Barton et al. 1995; Barton 2006; Heritage 2014; DeCasien and Higham 2019). A recent study modelling olfactory bulb evolution using extinct and extant taxa found evidence that the size of the olfactory bulbs (relative to the rest of the brain and absolute size) decreased in haplorhines and increased within the strepsirrhines (Heritage 2014).

The distinct difference in relative size of the olfactory bulbs between haplorhines and strepsirrhines is hypothesized to reflect differences in sensory specialization related to ecology in the two clades (Barton 2006; Heritage 2014). Ecological analyses suggest that the size of the olfactory bulbs (relative to the medulla, Barton 2006; and to the rest of the brain, Barton et al. 1995; DeCasien and Higham 2019) are significantly influenced by diet and activity pattern. Additionally, a negative correlation exists between visual and olfactory structures such that taxa with large olfactory structures tend to have smaller visual structures and vice versa depending on ecological condition. Specifically, nocturnal frugivores have larger olfactory structures and smaller visual structures while diurnal frugivores have larger visual structures and smaller olfactory structures (Barton et al. 1995; DeCasien and Higham 2019). Activity pattern may have played a major role in the variation of olfactory and visual structures between the two suborders as extant haplorhines are almost exclusively diurnal, and likely ancestrally diurnal (Kay et al. 1997; Ross and Kirk 2007) compared to the more variable activity patterns observed in extant strepsirrhines (Ankel-Simons and Rasmussen 2008).

Endocranial analysis of Paleocene and Eocene stem primates Plesiadapis cookei (Gingerich and Gunnell 2005), Plesiadapis tricuspidens (Orliac et al. 2014), Ignacius graybullianus (Silcox et al. 2009b), and Microsyops annectens (Silcox et al. 2010a) indicate that the size of the olfactory bulbs relative to endocranial volume are larger than in extinct and extant euprimates, but smaller than early eutherians (Kielan-Jaworowska 1984; Kielan-Jaworowska and Trofimov 1986) and apatemyids (Silcox et al. 2010b), and similar in size to fossil rodents and lagomorphs (Bertrand et al. 2016, 2017, 2018, 2019a; Bertrand and Silcox 2016; López-Torres et al. 2020; see Sect. 12.3.1). As noted above, of the plesiadapiforms only one taxon diverges from this pattern. The Early Eocene Niptomomys cf. N. doreenae (White et al. 2016), possesses larger olfactory bulbs (relative to endocranial volume) than other plesiadapiforms and stem rodents, and smaller olfactory bulbs than early apatemyids, suggesting it was more specialized for olfaction than other plesiadapiforms and stem rodents, but not compared to apatemyids (Silcox et al. 2011; Fig. 12.5c). Analyses of I. graybullianus and M. annectens found that the size of the olfactory bulbs relative to body mass, as opposed to endocranial volume, fell within the range of extant strepsirrhines (Silcox et al. 2010a). This result suggests that the size of the olfactory bulbs may have been relatively stable from the primate stem through the early evolution of euprimates and ultimately in the common ancestor of strepsirrhines, although they accounted for a smaller percentage of the brain (Harrington et al. 2016). However, it is unclear whether stem primates showed reduction in the relative size of the olfactory bulbs relative to the ancestral condition given the conflicting signals about the primitive states from apatemyids and members of Glires. As the expansion seen in the rest of the euprimate brain is often attributed to visual specialization (Barton 1998; DeCasien and Higham 2019), the proportionally large olfactory bulbs in stem primates (i.e., plesiadapiforms) suggest they relied more on olfactory signals than their extant relatives (Silcox et al. 2009b, 2010a; Orliac at el. 2014).

Among early euprimates (i.e., adapoids and omomyoids), the smallest olfactory bulbs relative to endocranial volume are found in the omomyoid Microchoerus erinaceus (Ramdarshan and Orliac 2016; Table 12.1), which could suggest that the grade shift in the relative size of the olfactory bulbs observed in extant strepsirrhines and haplorhines may have occurred early in the diversification of the two clades. However, the olfactory bulbs of M. erinaceus’ close relative, Necrolemur antiquus, are more similar in relative size to adapoids (Harrington et al. 2020; Table 12.1), making it less clear that the shift has an ancient origin. The onset of the apparent grade shift in relative olfactory bulb size that differentiates extant strepsirrhines and haplorhines is not clearly evident even in stem anthropoids. The olfactory bulbs of the stem anthropoid, P. grangeri, are large relative to both brain volume and body mass, within the range of extant strepsirrhines (Bush et al. 2004b). Similarly, early catarrhines (Victoriapithecus and Aegyptopithecus) possess relatively large olfactory bulbs, also within the range of extant strepsirrhines (Gonzales et al. 2015). In contrast, the earliest stem platyrrhine known from an endocast, Chilecebus carrascoensis, has small olfactory bulbs, smaller than the average for extant haplorhines (Ni et al. 2019). This suggests that the extreme reduction in the size of the olfactory bulbs in extant catarrhines and platyrrhines occurred independently (Heritage 2014; Gonzales et al. 2015; Ni et al. 2019) and not at the base of Anthropoidea, which is a powerful example of the importance of the fossil record to establishing the evolutionary context of evolutionary changes.

It is also unclear when or how the trade-off between visual and olfactory structures, observed particularly among anthropoids, occurred. For example, the stem anthropoid P. grangeri had large olfactory bulbs for a euprimate (Bush et al. 2004b) and large optic foramen areas and endocranial volume (Kirk 2006), which suggests it both retained the apparatus for strong olfactory abilities while also possessing adaptations for higher acuity vision. Phylogenetically controlled regressions of total visual input to the brain (measured using optic foramen area and orbit size) and olfactory bulb size relative to body mass in a sample of extant and fossil euprimates, including P. grangeri, Aegyptopithecus, and C. carrascoensis, failed to identify a significant correlation between the two, indicating that changes in olfactory and visual structures occurred independent of one another (Ni et al. 2019). Again, the inclusion of fossils re-frames conventional stories of evolutionary change within Primates.

Endocranial reconstructions of the inner ear, and particularly the semicircular canals, have also been used to investigate the connection between ecology and sensory capability in early Tertiary primates (Silcox et al. 2009a; Ryan et al. 2012; Bernardi and Couette 2017). The three arcs (anterior, lateral, and posterior) of the semicircular canals help detect the angle and velocity of an animal’s head movements. This information, alongside visual, proprioceptive, and otolithic information, is used to control body movements and stabilize gaze, functions suggested to be especially important for fast moving and arboreal animals (Spoor and Zonneveld 1998). The potential relationship between locomotor behavior and the semicircular canals was examined in a large sample of primates and mammals by Spoor et al. (2007). Multiple regression of average canal radius against body mass and locomotor agility indicated that fast, more agile species tend to have larger semicircular canals relative to body mass. Within primates, taxa with the smallest semicircular canals included slow quadrupedal arborealists (i.e., lorises) and large bodied great apes whereas taxa with the largest semicircular canals included specialized leapers (i.e., tarsiers and galagos) and acrobatic brachiators (i.e., gibbon).

Analysis of stem primates of the families Micromomyidae, Paromomyidae, Plesiadapidae, Carpolestidae, and Microsyopidae; adapoids of the families Adapidae and Notharctidae; and omomyoids of the families Omomyidae and Microchoeridae, found that the agility estimates from the semicircular canals were largely consistent with the reconstructions of locomotor behavior derived from postcrania (Silcox et al. 2009a; Bernardi and Couette 2017). Stem primates, adapids, and the primitive notharctid Cantius nuniensis had smaller semicircular canal radii relative to body mass, and therefore, were relatively slow-moving animals, a conclusion that is supported by postcranial material (when available) which suggests they were not specialized leapers. Omomyids and most notharctids, whose postcrania indicate occasional leaping, had relatively larger semicircular canals, similar to extant galagids, which engage in some leaping but are mostly arboreal quadrupeds. These analyses were unable to identify fine scale distinctions between locomotor behaviors within the stem primates (Silcox et al. 2009a), which reflects the ability of this method to only speak to relatively coarse differences in locomotor type. Ryan et al. (2012) assessed semicircular canal size and agility in anthropoids, reconstructing early anthropoids and catarrhines as being relatively slow moving, whereas early platyrrhines were more agile compared to earlier forms.

It is worth acknowledging that these analyses rest on a scale of agility scores that was generated entirely subjectively (Spoor et al. 2007). A much more rigorous, quantitative approach was taken by Malinzak et al. (2012), who took actual 3D vector measurements from a sample of primates while they were locomoting. These authors found that rotational head speed was more strongly correlated with the angles of the three semicircular canals (and how closely they approach orthogonality) than with their size. Unfortunately attempts to apply these methods to predictions of locomotion for fossil euarchontoglirans have failed to produce results that are consistent with what is known from postcranial data (Bernardi and Couette 2017; Bhagat et al. 2020), perhaps because the sample of modern animals was fairly narrow in scope (11 species, all strepsirrhines). Certainly, more data of this type would enhance our ability to probe the limits of semicircular canal data for inferring aspects of behavior in fossil taxa.

In recent years, several analyses have attempted to expand the scope of the data that can be used to examine fossil endocasts, and help understand the sensory significance of endocranial variation, by using geometric morphometrics and landmark based analyses (e.g., Pereira-Pedro and Bruner 2018). Notably, some of these studies have used sulci to delimit functionally specific brain regions in phylogenetically constrained groups (Kobayashi et al. 2018; Pereira-Pedro et al. 2019, 2020). Whereas this new method will be useful for analysis of recent fossil primates, its application to phylogenetically diverse groups with significant variation in sulcal anatomy, groups which contain fossils whose sulcal configuration is not well known, or lissencephalic species, has not been investigated in any fully published work (but see Makedonska et al. 2008; Allen 2014; Lang et al. 2019). Regardless, as this method continues to develop, there may be more information which can be gained from currently under-investigated aspects of sensory neuroanatomy in fossil primates, specifically related to taste, touch, and hearing. As new specimens emerge, and these new methods are developed, we will be able to expand and refine our understanding of primate sensory neuroanatomy.

12.4.4 Evolution, Form and Function of Derived Brain Structures

In addition to providing new perspectives on old questions, 3D data make it possible to ask a new range of questions based on the ability to more accurately quantify volumes for individual parts of the endocast, such as the olfactory bulbs (see Sect. 12.4.3) or petrosal lobules (Lang et al. 2018, 2022), as well as providing measures of surface areas. With respect to the latter, Jerison (2012) developed a method for measuring the relative size of the neocortical surface using laser scans of physical endocasts. He found a relationship between the degree of neocorticalization and time, with fossil euprimates standing out as always having larger relative neocortices than their contemporaries (Jerison 2012: fig. 6). Long et al. (2015) further elaborated on this method using X-ray CT data, and added values calculated from endocasts of plesiadapiforms (see also Orliac et al. 2014; Harrington et al. 2016; Ramdarshan and Orliac 2016). Although they found that Jerison’s conclusion was supported for euprimates, plesiadapiforms were inferred to be more like contemporary fossil non-primates in their degree of neocorticalization.

There was an issue with the dataset used by Long et al. (2015), however—it lacked any other euarchontoglirans. As such, it does not directly answer the question of whether or not there were shifts in relative neocortical surface area at the primate (vs. euprimate) node. Unfortunately, the endocast of Labidolemur kayi is not well enough preserved to indicate the location of the rhinal fissure. However, data on neocorticalization are available for the stem lagomorph Megalagus turgidus (López-Torres et al. 2020) and for various early rodents (i.e., ischyromyids; Bertrand and Silcox 2016; Bertrand et al. 2016, 2019a; Bertrand and Silcox this book). If the comparison is limited to the oldest and some of the most basal rodents for which there are quantitative data (i.e., members of the genus Paramys) and to Megalagus (as the only stem lagomorph for which there are data), then it does appear that early primates may have been slightly neocorticalized relative to primitive members of Glires (i.e., see Bertrand et al. 2016: fig. 6). However, if the comparative frame is expanded to include a broader range of ischyromyids, then the contrast is less clear, with their range of variation in the neocortical ratio overlapping the range known for plesiadapiforms (Bertrand et al. 2019a; see also López-Torres et al. 2020: fig. 4b). It would be helpful if quantitative data were available for a basal taxon that was not already a rodent or a lagomorph. Unfortunately, the known natural endocasts of Rhombomylus turpanensis (as the best-known candidate for this position) do not preserve the rhinal fissure (Meng et al. 2003; contrary to the impression provided by Orliac et al. 2014: fig 4). There are several nicely preserved crania of R. turpanensis (i.e., see Meng et al. 2003: fig. 26) so perhaps this issue might be solved by the CT-scanning and digital extraction of an endocast from one or more of them.

12.5 Future Directions: Outstanding Questions and Perspectives

There are three main directions that the study of early primate brain evolution using endocasts are likely to take in the coming years. The first relates to the comparative context for studying changes near the base of the primate tree. A lot of progress has been made in expanding the dataset relevant to assessing plesiomorphic states in Primates and Euprimates. This includes the first virtual endocasts for plesiadapiforms (Silcox et al. 2009b, 2010a; Orliac et al. 2014), which allow for high quality quantitative data to be captured and compared to the data from fossil and living euprimates. Also, very important has been the expansion of our knowledge of fossil members of primates’ close relatives (Silcox et al. 2011; Bertrand and Silcox 2016; Bertrand et al. 2016, 2019a, b; López-Torres et al. 2020). However, there remain critical holes in this sample. There are endocasts of plesiadapiforms that have not yet been published in full (Niptomomys cf. N. doreenae [White et al. 2016]; Carpolestes simpsoni [Silcox et al. 2017b]; Ignacius graybullianus [Boyer et al. 2011; Long et al. 2015]; Plesiadapis tricuspidens [Kristjanson et al. 2016]). But even when these specimens are published, it will still be the case that all plesiadapiform endocasts are known from relatively derived members of their respective families, and from branches several nodes from the base of the primate tree. It would be a tremendous boon to add a more basal plesiadapiform (e.g. a pugatoriid or palaechthonid) to the sample, beyond the very limited information that can be gleaned from Torrejonia wilsoni (Chester et al. 2019: fig. 3). Additional data for early Euprimates would also be beneficial. Extracting virtual endocasts from specimens that have already been studied in the context of early primate brain evolution (e.g., Leptadapis magnus, Pronycticebus gaudryi, Tetonius homunculus) would be an obvious first step. There is also an abstract published that mentions endocranial data for several additional specimens of European adapoids (Makedonska et al. 2008), but that study has never been published in full. However, even with these additions we would still be lacking endocranial data for the most basal adapoids and omomyoids (i.e., Teilhardina, Cantius, Donrussellia), which would be beneficial to characterizing the primitive states for these groups. As noted above, data for early omomyoids would be particularly valuable for assessing the timing of changes in relative brain size near the base of the primate tree. There are some cranial specimens known for some of these genera, which may be able to provide at least select endocranial details (e.g. Rose et al. 1999; Ni et al. 2004).

Beyond primates, it would be beneficial to have additional data for early members of Euarchontoglires. As discussed above, Rhombomylus turpanensis is one obvious candidate for this, and additional data for apatemyids would also be of interest (e.g., quantitative data for the endocast of the derived apatemyid Carcinella sigei; von Koenigswald et al. 2009). But probably even more exciting would be data for other fossil groups of Euarchontoglires, which could add additional perspectives on primitive states for that group (e.g., mixodectids, anagalids). Finally, having data for more than a single stem lagomorph would be crucial, particularly since Megalagus turgidus is early Oligocene in age (López-Torres et al. 2020), and so notably more recent than the primate taxa under discussion here. Obviously, the situation is much better for early rodents (Bertrand and Silcox 2016; Bertrand et al. 2016, 2019a), with both a larger number of endocasts, and a greater temporal depth, extending back to the early Eocene. However, the oldest endocast for which good quantitative data are available is from Wa7 (Wasatchan North American Land Mammal Age 7, ~52.4–50.1 mya; Bertrand et al. 2016; note that there is an endocast for a specimen from Wa6 [Notoparamys costilloi] but it is too compressed to provide good quality quantitative data; Bertrand et al. 2019a), which is several million years after Rodentia entered North America at the start of the Clarkforkian (Rose 1981, 2006; Korth 1994) and so certainly well separated in time from the origin of the order. In sum, further understanding the primitive context of primate evolution requires not only a better sample for early primates, but also for relevant outgroup taxa.

Second, future work will likely enhance our knowledge of intraspecific variation in fossil primate taxa. The best samples currently known are for adapoids, although the maximum sample size for any one taxon is only N = 5 for Smilodectes gracilis (Gazin 1965; Harrington et al. 2016), of which one (UM 32773) is subadult. Nonetheless, it would be of value to study this sample through the lens of intraspecific shape variation. Apart from the North American notharctid adapoids, the best candidates for understanding variation in closely related taxa are probably the large bodied European adapoids, known from numerous three-dimensionally preserved crania (e.g., Godinot and Couette 2008; Makedonska et al. 2008). A complication to such studies is the confounding effect of body mass, in light of the divergent estimates from different equations (e.g., see Harrington et al. 2016). The ideal situation is for the cranial specimen from which the endocast is extracted to be associated with postcranial material, allowing for a completely independent body mass estimate, but there are only a very few instances in which this is the case (e.g., Plesiadapis cookei [Gingerich and Gunnell 2005; Boyer and Gingerich 2019]; one specimen of Smilodectes gracilis [USNM V 17994; Harrington et al. 2016]).

Third, and finally, much of the discussion about endocranial variation in early Primates has focused on size, so an area of future growth is to expand our understanding of variation in shape. There have been some analyses mentioned in abstracts or unpublished theses that have looked at shape variation using geometric morphometric methods (e.g., Makedonska et al. 2008; Allen 2014; Lang et al. 2019), but these studies have not yet been published in full. A critical element in the interpretation of such analyses is the degree to which differences in shape can be interpreted with respect to function. Although there is obviously a very large literature on functional aspects of the brain in Primates, integrating the details of this literature with the data that can be observed or measured from an endocast is an ongoing challenge. For example, studies of function in the cerebral cortex that require the brain to be flattened for examination can be difficult to translate into the three-dimensional surface of an endocast. Application of new 3D imaging techniques such as DICE-CT may help in successfully integrating functional data with the type of shape data available for endocasts, permitting more direct inferences to be made about differences among endocasts from fossil taxa.

12.6 Concluding Remarks/Final Considerations

The last 15 years have seen a renaissance in the study of endocranial morphology in early primates, largely spurred by the growing availability of high-resolution X-ray CT scanners. The endocasts that have emerged for early primates have confirmed some historical perspectives, answered some questions, but also spurred some new research directions. In spite of their areas of disagreement, Jerison and Radinsky did agree on some major elements of the interpretation of the pre-CT record of early euprimate endocasts, including the evidence for likely expansion of the temporal and occipital lobes and development of the Sylvian sulcus, and the presence of a less developed frontal lobe than in living primates. As discussed in Sect. 12.2.1, some of the elements that fueled their debates have effectively been answered by the data made available by CT. It is no longer necessary to rely on methods that make major assumptions about the shape of the brain to derive an estimate of volume. And we now have some good quality quantitative data for the first radiation of fossil primates, which show that the distinctively primate-like traits of euprimates actually do not characterize the first members of the order. However, as discussed in Sects. 12.2.2 and 12.4.2, the appropriate context for looking at relative brain size continues to be problematic, and more data are needed to help interpret the patterns seen in the endocasts we do have.

Although this chapter has by necessity focused mainly on the record of early primates and euprimates, there is also a burgeoning record of virtual endocasts for later non-hominid primates (e.g., Bush et al. 2004a,b; Simons et al. 2007; Ryan et al. 2008; Kay et al. 2012; Allen 2014; Gonzales et al. 2015; Beaudet et al. 2016; Ni et al. 2019). A few generalities about the broader picture of primate brain evolution are emerging from these studies, which have relevance to the interpretation of the earlier fossils. As discussed above, there is growing evidence for rampant parallelism in brain size evolution among lineages of fossil primates (Sect. 12.4.2, Table 12.2 and Fig. 12.6; see also Allen 2014; Gonzales et al. 2015; Ni et al. 2019). It is worth noting these parallel expansions make analyses of primate brain size evolution that do not integrate fossils quite problematic. Such analyses generally assume the process of brain size change will follow the most parsimonious or likely path over the entirety of the 65+ million years of primate evolution. The fossil record suggests that it does not.

There is also evidence from a few points in the primate tree that changes in form precede changes in size (Allen 2014; Gonzales et al. 2015; Ni et al. 2019). So, for example, the endocast of the stem cercopithecoid Victoriapithecus had already evolved the pattern of gyrification characteristic of cercopithecoids, but at a small endocranial volume (Gonzales et al. 2015). This finding parallels the conclusion of Harrington et al. (2016) that the major structural changes associated with the euprimate node preceded significant relative change in brain size, although as noted above this conclusion was critiqued by Gilbert and Jungers (2017). In our opinion, we cannot reach an unambiguous answer on this point with the currently available fossil record (see discussion in Sect. 12.4.2).

Interpreting these patterns in an adaptive context poses a final challenge to our understanding of brain evolution. There are various factors that have been identified as critical to the process of brain evolution in the order including the evolution of visual processing (Barton 1998; Kirk 2006), the importance of social behavior (Dunbar 1998; Dunbar and Schultz 2007), the necessity of processing complex information from the arboreal environment (Falk 2007), and the impact of variation in diet (Harvey et al. 1980; DeCasien et al. 2017). The paleoneurological data have provided some possible insights into these competing influences on primate encephalization. In particular, the fact that plesiadapiforms, who were arboreal, exhibit plesiomorphic endocranial features suggests that moving into the trees did not have a marked impact on the form or size of the brain (Silcox et al. 2009b, 2010a). In contrast, improvements associated with visual processing were likely critical to at least some major transformations in the primate brain (Kirk 2006; Silcox et al. 2009b, 2010a). Studying the impact of other factors, such as diet, in a context that includes data from fossil primates has the potential to enrich our understanding of the reasons behind change over the course of primate brain evolution.