Introduction

At the root of the conflict between the sexes lies a fundamental sex difference in reproductive investment and optimal mating rates. Trivers (1972), building on earlier work by Bateman (1948), pointed out that differential reproductive investment should result in conflict between the sexes due to different optimal mating rates. The more limited sex (often the female due to higher reproductive investment per gamete) has a lower optimal mating rate than the less limited sex (often the male, particularly if sperm is all that he is offering). If females mate above their optimal mating rate, the costs associated with multiple matings can lower her lifetime reproductive success (Rice 1996; Holland and Rice 1999; Arnqvist and Rowe 2002, 2005; Parker 2006). There are costs of multiple matings for males; however, the potential gain to a male’s fitness can be quite high. This disparity in optimal mating rates between the sexes has been termed sexual conflict (Parker 1979) and it can lead to a coevolutionary arms race between the sexes.

This sexually antagonistic coevolution has generated a variety of sexually dimorphic morphological and behavioral adaptations. Males and females have diverged not only in the reproductive organs necessary for fertilization, but also in specialized hardware that enables each sex to exert some control over mating rate. For males, sexual armaments can provide an edge in coercing a female into mating more often than her optimal rate. For females, counter adaptations in the form of sexual armor can slow or control the rate of matings (reviewed in Brennan and Prum 2014). Behavioral adaptations have diverged between the sexes that either lead to increase conquests for males or decrease male exposure for females (reviewed in Arnqvist and Rowe 2005; Parker 2006; Magurran 2011; Rosenthal 2017). Yet, the consequences of this pervasive sexually antagonistic coevolution between the sexes quite likely extend beyond the external phenotype and into the internal arena where these behavioral differences are regulated.

One of the relatively unexplored, but highly probable, consequences of this fundamental process of sexual selection is that sexual conflict impacts cognition differently between the sexes. Cognitive processes control behavioral output and as behavioral strategies diverge between the sexes, we can expect that associated cognitive mechanisms diverge with them (Jones et al. 2003). For instance, meadow vole males with larger home ranges than females have greater spatial memory abilities (Gaulin and Fitzgerald 1989). And cowbird females who have larger home ranges than males have a larger hippocampus (Sherry et al. 1993), a brain region associated with spatial memory. Meanwhile, male and female sticklebacks have different brain sizes and different parental roles, suggesting that parenting may require more from their nervous system than non-parental behavior (Kotrschal et al. 2012). These examples illustrate that if males and females have discrete roles, their brains and cognitive capacity may also differ in specific ways. These examples highlight how sex-specific roles influence cognitive hardware, but they do not address how sexual conflict itself drives differences in cognition. Here, I focus on how sex-specific roles or behaviors driven by sexual conflict may contribute to sexual divergence in behavior, cognitive performance, and the behavioral associations linked to cognition.

To date, probably the best known vertebrate group to explore the influence of sexual conflict on cognition is the group of freshwater livebearing fishes known as the poeciliids (family Poeciliidae). Poeciliids rely on internal fertilization with large reproductive investment differences between the sexes, as females have internal gestation (from 20 to 32 days) and males provide only sperm (Parzefall 1973; Constanz 1989; Ryan and Rosenthal 2001). The intense intrasexual competition amongst males has led to the evolution of elaborate reproductive structures (e.g., features that allow male intromittent organs to attach and insert into the female gonopore and/or remove competitor’s sperm; Langerhans 2010, 2011; Kwan et al. 2013; Jones et al. 2016). Importantly, we see that both males (Mautz and Jennions 2011) and females (Dadda 2015) have evolved decision-making processes during sexual encounters that can increase their respective reproductive success. In addition, poeciliids have become a model system to test for cognitive variation (Bisazza et al. 1997; Laland and Reader 1999a, b; Agrillo et al. 2017). Hence, they are a prime system to directly explore how sexual conflict drives divergent behavioral repertoires between the sexes and its impact on cognition.

Sexual conflict in Poeciliidae

Poeciliids have some of the highest mating rates amongst vertebrates, as well as some of the greatest relative differences in sex-specific optimal mating rates. Males provide only sperm during reproductive events and have no resources to defend (Endler 1987; Houde 1997). Over half of the more than 200 poeciliid species represent mating systems dominated by male coercion (sneaking up on females from behind with unsolicited gonopodial thrusts), whereas the other taxa include both courtship and coercive tactics (Farr 1989; Bisazza 1993; Magurran 2011). Meanwhile, females have a month-long gestation period and are rarely sperm-limited due to their capacity to store sperm (Winge 1937; Constanz 1989). The consequence is an operational sex ratio that is heavily male biased and optimal mating rates that differ by orders of magnitude between the sexes. In theory, the optimal mating rate for males is several times a day; whereas for females, it is less than one mating event per month. Further intensifying the sexual conflict scenario is the last-sperm precedence of this system (Evans and Magurran 2001), providing more incentive for males to engage in multiple matings with individual females. Given these dynamics, it is perhaps not surprising that male mating attempts are consistently high across poeciliid taxa ranging from greater than one attempt per minute in some guppy populations (Magurran and Seghers 1994a) and Gambusia species (G. affinis; Smith 2007) to variable (0.25–5/min) across reproductive strategy in Xiphophorus nigrensis (Ryan and Causey 1989; Wong et al. 2011). In general, activity budgets for males reveal a substantial portion of their day is engaged in mating attempts (50%; Magurran and Seghers 1994a).

Female activity budgets, meanwhile, are heavily skewed towards foraging and avoiding male harassment (Houde 1997; Magurran 2011). Avoiding male encounters may reduce the direct costs of mating with males, as the hooked and barbed male gonopodium can cause injury to females (Clark et al. 1954; Constanz 1989; Greven 2005). Female guppies have been found to spend six times as much time foraging than male guppies (Dussault and Kramer 1981). This large difference in foraging effort between the sexes is driven by sexual dimorphic relationships between fecundity and size. Unlike males, poeciliid females have indeterminant growth, and their fecundity is directly linked to their size (Reznick 1983; Abrahams 1989; Booksmythe et al. 2016). Hence, females prioritize foraging opportunities over mating opportunities. However, the high rate of male harassment in poeciliids can lead to interference in female foraging. Male sexual harassment in poeciliid taxa consistently leads to a reduction in female foraging efficiency (Poecilia reticulata, Magurran and Seghers 1994b; Houde 1997; P. latipinna, Schlupp et al. 2001; G. holbrooki; Pilastro et al. 2003; Dadda et al. 2005).

Female behaviors evolving in response to sexual conflict

Many female poeciliids have resorted to increasing their shoaling or aggregation behavior with other females as a strategy to reduce male harassment and harassment-induced reductions in foraging (Pilastro et al. 2003; Dadda et al. 2005). Recent comparative research suggests that the strength of female aggregation response is a predictable function of the coercion efforts by conspecific males (Dadda 2015). Specifically, females that experience a high degree of sexual conflict (high levels of gonopodial thrusting in G. holbrooki and P. reticulata) exhibited a greater response to shoal together in the presence of male conspecifics relative to being presented with conspecific females (Dadda 2015), whereas females in the Xiphophorus genus which are exposed to relatively fewer coercion attempts by male conspecifics (X. hellerii, X. mayae) showed no change in aggregation response in the presence of males (Dadda 2015). Yet, this response to aggregate with other females may also induce some costs of its own. Controlled laboratory studies have shown that female western mosquitofish (G. affinis) at high densities suffer from lower reproductive success than those at low densities (possibly through female competition and increased cannibalism; Smith and Sargent 2006; Smith 2007).

The fundamental process of negotiating this sexual conflict in poeciliids leads to a variety of direct and indirect behavioral differences between the sexes (Magurran and Maciás Garcia 2000). Female tendency to shoal more than males is just one of the many sex differences in behavior that are indirectly linked to sexual selection and sexual conflict. Male harassment can lead to differentiation in habitat use, with females occupying higher predation risk areas to avoid male harassment (Croft et al. 2006; Darden and Croft 2008). Furthermore, male and female poeciliid often approach predatory encounters differently. Female guppies are more sensitive than males to predators, detecting them earlier and conducting more inspections (Magurran and Nowak 1991; Magurran and Seghers 1994a).

Male behaviors evolving in response to sexual conflict

With the evolution of female matrotrophy and higher female investment in reproduction, the disparity in optimal mating rates between the sexes set the stage for intense sexual conflict (Fig. 1). Phylogenetic analyses have confirmed that as females invest more in reproduction (specifically the evolution of a placenta), the number of taxa with predominantly coercive males with small body sizes increases (Pollux et al. 2014). Smaller male bodies are predicted to be favored as a means for more effective sneak copulations as they are less easily detected by the female during their rear approach. Intense sexual selection is presumed to play a role in the higher diversifying effect on morphology for males relative to females (Culumber and Tobler 2017), as well as great behavioral diversification. While male coercion tactics are expected to intensify as an outcome of sexual conflict, another potential outcome is the evolution of alternative reproductive male behavioral strategies such as courtship (e.g., Wang et al. 2015). Specifically, a phylogenetic analysis of the behavioral and gonopodial features of 10 Gambusia species suggests that as species invest more heavily in gonopodial features associated with sexual conflict, behavioral displays associated with courtship also emerge (Wang et al. 2015).

Fig. 1
figure 1

Predicted relationships between reproductive investment, sexual conflict, sexually dimorphic behaviors, and cognition in Poeciliid fishes. (a) Poeciliid taxa with matrotrophic females (teal green color) invest significantly more energy per reproductive event (greater gamete size, internal gestation periods) than male conspecifics (blue color). (b) Difference in relative investment per reproductive event leads to different optimal mating rates between the sexes with males having the higher (+) optimal mating rate than females (−). Optimal body size for fecundity and/or mating success maximization can also differ between the sexes. Larger body sizes (+) usually favor poeciliid female fecundity (Booksmythe et al. 2016) while smaller body sizes (−) are often associated with higher male mating success in coercive species (Bisazza and Pilastro 1997; Pilastro et al. 1997; Pollux et al. 2014; Head et al. 2017). Note, the * indicates the relationship between male size and mating success can be variable. Studies within G. holbrooki have found presence (Head et al. 2017) and absence (Booksmythe et al. 2016) of evidence for a smaller male mating advantage. Furthermore, it is important to note that numerous poeciliid taxa contain an alternative large, courting male phenotype that is preferred over the small male by females during mate choice encounters. (c) Differences in life history strategies and sexual selection pressures manifest in sexually dimorphic behaviors. Females shoal more and spend significantly more time foraging than males (see Table 1). Both of these behavioral differences are demonstrably linked to sexual selection as females can increase their reproductive success by investing in greater body size/fecundity and avoiding male harassment via shoaling. Other sexually dimorphic behaviors demonstrated in some poeciliid taxa are also predicted (gray box) to be driven by sexual selection including greater exploratory behaviors by males and greater vigilance/anxiety by females (see Table 1). (d) Behavioral differences between the sexes are expected to contribute to variation in cognition. Studies have found sex differences in learning performance (spatial learning) and cognitive style (decision speed) as well as learning innovation (reversal learning, detour reaching) with poeciliids (see text for further discussion)

Behaviorally, male poeciliids tend to be more active, disperse further, and explore more than females (Table 1). These behavioral differences are predicted to stem from sexual selection pressures on males to maximize their reproductive success by searching for more mates. The pressure on males to find more mates is presumed to be responsible for higher emigration rates in male relative to female guppies (Croft et al. 2003). It is also considered a contributing factor to the difference in anti-predatory responses between the sexes. In high-predation guppy populations, males take advantage of greater female inspection behaviors by engaging in sneaky copulations while females engage in predator inspections (Magurran and Nowak 1991).

Table 1 Sexually dimorphic behaviors amongst some Poeciliids (M refers to males; F refers to females)

The intense sexual selection that is present in poeciliids has resulted in behaviors that directly influence control over mating success, as well as behaviors that can be seen as having indirect effects on this process. The evidence to date shows that poeciliid males and females differ in movement, foraging, predatory response, mating motivation, shoaling, or aggregation tendencies. In the sections below, I will attempt to discuss how these differences caused by sexual conflict have consequences (or implications) for cognitive variation between the sexes (Fig. 1).

Sexual conflict and sex-specific cognitive styles or cognitive-behavioral profiles

Each of the behavioral sex differences discussed above has interesting implications for the emergence of potential sex differences in cognitive style and learning attributes (Lucon-Xiccato and Bisazza 2017a). Differences in individual motivation for foraging (Laland and Reader 1999a), mating (Bisazza et al. 1998), or tendency to explore (Guillette et al. 2009) have all been associated with individual differences in learning performance or cognitive style. Cognitive processes can be evaluated in terms of how well individuals learn to associate a specific reward or punishment with an outcome (associative learning) and how well they are able to incorporate new information to update their decision rules (reversal learning), as well as how innovative or flexible they are in solving problems or learning challenges (innovation, learning flexibility). In addition, researchers have recently noted that there are distinct differences in how individuals behaviorally approach a learning task (cognitive style). Cognitive style refers to the phenomenon that two groups of individuals may all learn a specific associative learning task, but they may do so with completely different approaches to the problem. For example, some individuals may be very fast in making decisions during the learning or training trials, whereas others may be more slow or cautious during learning trials. Lastly, researchers have begun to characterize individual’s behavior outside the learning assay and have noted that more general behavioral characteristics can be predictors of performance within a learning assay (e.g., cognitive-behavioral profiles, Etheredge et al. 2018).

Male and female poeciliids have different motivations to feed, find mates, and shoal. Hence, poeciliid learning assays that focus on foraging, finding mates, or shoaling often find sex differences in performance (Fig. 1). When given a novel foraging task involving spatial exploration, female guppies more than males were likely to innovate and problem solve (Laland and Reader 1999a). In a learning flexibility task known as detour reaching where individuals have to move away from a target (move around a transparent barrier) before successfully reaching the goal (shoaling partner), female guppies solve the problem faster than males (Lucon-Xiccato and Bisazza 2017b). These results appear to corroborate previous results revealing that female guppies were faster at reversal learning than male guppies (Lucon-Xiccato and Bisazza 2014). Meanwhile, male guppies were found to be faster at learning a complex spatial task than females (Lucon-Xiccato and Bisazza 2017b) and made faster decisions (but not more accurate ones) than females in a suite of visual discrimination tasks (Lucon-Xiccato and Bisazza 2016). Taken together, these sex differences in performance on sex-specific tasks (foraging innovation and spatial memory) appear consistent with sex differences in reproductive strategy. Males are known to emigrate more than females, and females emphasize foraging more than males. The greater cognitive flexibility observed in females and the greater decision speed in males may possibly have their roots in sexual conflict dynamics between the sexes, but much further work is required to test such a hypothesis.

One arena of cognitive-behavioral relationships where male and female poeciliids reveal an intriguing contrast involves how the sexes relate to threatening situations. As mentioned above, male and female guppies appear to have different sensitivity to predation threats. Having a sexually dimorphic response to predation threat may lead to differences in learning styles in poeciliids. Burns and Rodd (2008) compared spatial decision-making processes between high- and low-predation guppy populations. They found more wary guppies from high-predation sites took longer to make decisions than guppies from low-predation areas. Recent intrapopulation research found a sexually dimorphic relationship between exploratory behaviors and learning in the western mosquitofish, G. affinis (Etheredge et al. 2018). In this study, males that explored more in novel and threatening contexts performed better on a visual discrimination task (numerosity discrimination), whereas females showed a negative or neutral relationship between exploration and learning performance (Fig. 2). These results are consistent with prior research in male guppies showing a positive relationship between predator inspection and associative learning performance (Dugatkin and Alfieri 2003), but differ from observations in female Brachyrhaphis episcopi where a positive relationship between exploratory tendencies and associative learning performance was found (DePasquale et al. 2014). Clearly, a more comprehensive examination of sexual differences across species is warranted.

Fig. 2
figure 2

Male and female G. affinis have distinct relationships between associative learning performance and anxiety/exploratory tendencies. Male and female learning and exploratory tendencies were evaluated with independent assays in G. affinis (Etheredge et al. 2018). Fish were first evaluated for learning performance with a numerosity discrimination assay involving two days of habituation in an experimental arena, four days of training (food reward associated with either the higher or lower number of geometric shapes on a screen), followed by three days of testing (no reward) with novel numerical contrasts. Each fish’s learning score was calculated across test trials as the average time spent near the trained contingency (the higher or lower number of shapes rewarded during training). Anxiety and/or exploration tendencies were evaluated in two distinct contexts. Thigmotaxis (favoring edges and corners) behavior was scored in each fish’s inaugural numerosity training trial as an anxiety measure in a relatively new environment. Anxiety and exploratory behavior were also evaluated the day after numerosity testing was completed with a scototaxis trial. Scototaxis (favoring shadows), a common anxiogenic assay in small fish (Maximino et al. 2010) similar to the light-dark test in rodents, involves placing a fish in a novel tank environment that is 50% dark (half the tank lined in black felt) and 50% light (half the tank lined in white felt). In scototaxis trials, time spent in the dark side is interpreted as a measure of anxiety, while time spent on the white/light side is interpreted as a measure of boldness or exploration. We then tested for differences between the sexes by examining the difference in sex-specific slopes between anxiety/exploration score and learning score. The relationship (± 95 CI represented in gray) between Gambusia affinis male (blue) and female (purple) learning performance (in a numerosity discrimination task) and exploration in a threatening and novel environment (left: time in white during a scototaxis trial; right: thigmotaxis during numerosity tank during day 1) are shown. Male and female G. affinis had significantly different slopes (* denotes p < 0.05) for learning performance and exploratory tendencies in these different contexts (figure redrawn with permission from Etheredge et al. 2018)

An initial exploration of how multiple behaviors implicated in sexual conflict are linked to cognition reveals sexually dimorphic relationships. Etheredge et al. (2018) examined how shoaling tendencies, activity levels, anxiety and exploratory responses are related to associative learning performance in a visual discrimination task in male and female western mosquitofish (G. affinis). We found that the sexes did not differ in learning accuracy (similar numbers of learners and non-learners between the sexes), but did differ in the suites of behaviors that predicted learning performance (Fig. 3). High-performance learner males exhibited different behavioral attributes than high-performance learner females, with females characterized by a suite of behavioral traits (including sociability which is a key female response to conflict) and males by only sociability. Whether these sex differences are a consequence of the sexual conflict in this species is not yet known, but further work within this system and comparative work across poeciliids with shared or divergent levels of sexual conflict will help ascertain the source of these relationships.

Fig. 3
figure 3

Divergent cognitive-behavioral profiles between the sexes in Gambusia affinis emerging from a numerosity discrimination task (Etheredge et al. 2018). Male and female G. affinis were trained with a food reward and tested on a numerosity discrimination task as described in Fig. 2. Fish were categorized as non-learners (orange) if they failed to meet the learning criterion (spending a > 60% median time with trained target (high or low side)): low-performance learners (gray) if they met the learning criterion in at least one of the test ratios but failed to show non-random performance across all 9 tests, and high-performance learners (green) if they met the learning criterion and showed significantly higher than chance performance across all 9 trials. Subsequent to this associative learning task, individual fish were tested on a battery of behavioral assays including sociability (shoaling tendency), activity (total movement during shoaling assay), and scototaxis (variety of anxiety and exploration measures recorded in a novel tank that was half white (threatening) and half dark (non-threatening)). A PCA was used to reduce the six anxiety- and exploration-related behaviors (two of which are found in Fig. 2) into two principle axes (Anxiety/exploration PC1 and Anxiety/exploration PC2). The discriminant function analysis for males (a) and females (b) using individual behavioral inputs (sociability, activity, anx/Exp PC1, anx/Exp PC2) revealed that the high-performance learning group was significantly differentiated from the non-learning group (in males) and both non-learners and low-performance learners (in females). The X axes represent the primary canonical axis resulting from the discriminant function analysis (male DFA1 in (a); female DFA1 in (b)), along with the (%) of behavioral variation explained by each axis. The Y axes refer to the secondary canonical axes (DFA2). Colored circles represent the normalized 95% confidence ellipses for each learning group, with the lack of overlap indicating significant differentiation due to variation in behavioral inputs. Furthermore, the behavioral inputs that predicted high-performance learning were different between males and females. Male high-performance learners exhibited greater sociability scores than males in other learning groups, whereas female high-performance learners were differentiated from other females via a combination of activity and sociability (and the two anxiety/exploration scores to a lesser degree). Figure redrawn with permission from Etheredge et al. 2018

Brain size and sexual conflict

One of the most compelling pieces of evidence that sexual conflict influences cognition is the artificial selection experiments with poeciliid species showing that as you select for one of these (conflict or cognition) you affect the other. Kotrschal, Kolm, and colleagues have undertaken an ambitious project of selecting on guppy (Poecilia reticulata) brain size. They created artificially selected lines of guppies for larger and smaller brain sizes. After two generations, their artificial selection efforts produced a 9% difference in brain size between the large and small lines (Kotrschal et al. 2013), and more interestingly, a suite of physiological, behavioral, and cognitive processes that differed between these lines. Perhaps most intriguing, with respect to this review, is that they found a suite of sex-specific effects of increasing brain size.

For females, but not for males, a larger brain led to an increase in associative learning performance in a numerosity discrimination task (Kotrschal et al. 2013), improved survival while co-habitating with a predator (Kotrschal et al. 2015a), and improved ability to discriminate among ornamented males (Correl-Lopez et al. 2017). Meanwhile, guppy males that were selected for larger brains showed an increase in male genital size and male coloration (Kotrschal et al. 2015b), and a faster ability to find mates in a spatial search task (Kotrschal et al. 2015c) relative to the smaller brained males. Hence, selecting for brain size influenced cognitive traits, behaviors, and the physiology involved in the sexual conflict dynamic between the sexes.

Further evidence of the link between sexual conflict and cognition was provided when this research team examined the results of artificial selection on genital size in the highly coercive eastern mosquitofish, G. holbrooki (Booksmythe et al. 2016). While artificially selecting for long versus short gonopodial length in G. holbrooki revealed no difference in male brain size, the researchers found an increase in female brain size in the longer gonopodial lines (Buechel et al. 2016). The researchers proposed that increasing cognitive processes was required in high sexual conflict environments (males with longer gonopodia) to enable females the behavioral flexibility to avoid male coercion. Taken together, these two different artificial selection approaches strongly suggest a selective association between sexual conflict and cognitive properties in poeciliids.

The artificial selection experiments with guppies and eastern mosquitofish might also provide some insight into the sex-specific cognitive performances observed in other poeciliid studies (Fig. 1d). If selecting for larger female brains leads to increases in associative learning, mate choice discrimination capabilities, and predator evasion, does a larger brain confer other cognitive benefits for female poeciliids? While there are only a few studies that examine poeciliid cognitive flexibility, it is noteworthy that females consistently outperformed males in different measures of cognitive flexibility (reversal learning, detour reaching, and learning innovation; Fig. 1d). This female-biased performance in cognitive flexibility for assays involving shoaling or foraging may be driven by sexual dimorphic responses for these specific tasks as mentioned above, or they may be a product of larger absolute brain size. Since females in many poeciliids have larger body sizes than male conspecifics, they are likely to also have a larger absolute brain size. The fact that larger brains often results in greater cognitive abilities (MacLean et al. 2014) could suggest that some of the increased cognitive abilities may be an indirect effect of size and not a direct consequence of pressures associated with sexual conflict.

Brain pathways and sexual conflict

While the artificial selection experiments with poeciliids suggest a strong relationship between the hardware components of sexual conflict and cognition, we still have an incomplete understanding of which parts of the brain and molecular pathways regulate sexual conflict interactions. Some early clues are emerging from research contrasting the behavioral genomic responses between mild (X. nigrensis, a northern swordtail) and high (G. affinis, the western mosquitofish) sexual conflict species. This early research suggests that molecular pathways are differentially engaged when females interact with high vs low sexual conflict encounters, that these plastic responses may be conserved across species, and sexual conflict interactions involve brain regions and molecular pathways regulating learning, memory, emotional processing, and social cognition.

Our laboratory began examining the neural correlates underlying sexual conflict encounters with a neurogenomic approach with female Xiphophorus nigrensis (Cummings 2012, 2015). X. nigrensis has three alternative male phenotypes that have genetically predetermined behavioral strategies: a low sexual conflict phenotype that only engages in courtship (large males with ornaments), a high sexual conflict male phenotype that engages in only coercion (small size, unornamented males), and a presumably intermediate sexual conflict phenotype that has a mixed strategy (intermediate-sized males that employ both mating tactics). With the courting phenotype, female X. nigrensis can exert more control over their own mating rate (they choose to cooperate with copulation or decline). Meanwhile, when females interact with the small coercive phenotype, they have less control. Not surprisingly, decades of behavioral research with this system has shown that females overwhelmingly prefer large class (courtship only) males over the small coercers (Ryan and Rosenthal 2001; Cummings and Mollaghan 2006; Wong et al. 2011).

We initially performed an exploratory neurogenomic experiment using a brain-specific microarray to identify candidate genes associated with mate choice decision making with X. nigrensis females (Cummings et al. 2008). When females were deciding between a high (coercer) and low (courter) conflict male, genes associated with synaptic plasticity were differentially expressed relative to other social conditions. Synaptic plasticity genes are functionally linked to learning. They are expressed by neurons to strengthen, initiate, or rewire connections to other neurons at the level of the synapse (a physiological necessity for learning and memory). We then employed targeted gene expression approaches with these candidate genes (using quantitative PCR) and found that expression of these genes at the whole brain level predicted the variation in female preference behavior for the courting phenotype over the coercer (Fig. 4a, Cummings et al. 2008; Lynch et al. 2012; Ramsey et al. 2012). Furthermore, we were able to disrupt female preference behavior in female X. nigrensis by blocking this candidate synaptic plasticity pathway (Ramsey et al. 2014).

Fig. 4
figure 4

Expression of synaptic plasticity genes (neuroserpin, neuroligin-3) exhibits contrasting patterns in poeciliid females exposed to high and low sexual conflict encounters. Individual variation in X. nigrensis female preference for courting male conspecifics over their coercive male conspecifics (a) positively covaries with synaptic plasticity expression in whole brain (filled in circles), and (b) localized brain regions (open circles) including the fish homologue to the human hippocampus (Dl). Meanwhile, in G. affinis, these same genes show (c) negative whole brain gene expression patterns with female preference behavior towards their coercive conspecific males (filled squares). (d) However, when G. affinis females are exposed to heterospecific courting males (large Poecilia latipinna), they show a positive correlation between synaptic plasticity expression (neuroserpin (shown), egr-1 (not shown)) and preference behavior (open triangles), and a negative correlation between synaptic plasticity expression (neuroserpin (shown), egr-1 (not shown)) and preference behavior (open squares) when exposed to coercive heterospecifics (small Poecilia latipinna). Data from Cummings et al. 2008; Lynch et al. 2012; Wong et al. 2012; Wang et al. 2014

Using in situ hybridization techniques, we found positive correlations between female preference behavior for large courting phenotypes and synaptic plasticity expression within brain regions associated with learning and memory in the teleost forebrain (Fig. 4b). We identified predictive patterns of female preference behavior from synaptic plasticity expression levels in brain regions associated with learning, emotional processing including the fish homologue of the mammalian palial hippocampus (Dl, dorsolateral telencephalon, see Fig. 4b), the basolateral amygdala (Dm, dorsomedial telencephalon), the preoptic area (POA), the ventral hypothalamus (HV), and the ventral nucleus of the ventral telencephalon (Vv) (Wong et al. 2012; Wong and Cummings 2014). These brain regions are part of a phylogenetically conserved network termed the social decision-making network (SDMN, O’Connell and Hofmann 2011, 2012) that involves brain regions responsible for collecting information, assigning salience and valence to different social agents, and coordinating an adaptive response. Our results showing predictive covariation between learning gene expression in some of the SDMN nodes suggest that female X. nigrensis are dynamically processing information (or retrieving relevant memories) during encounters with different male phenotypes to coordinate their behavioral response.

Comparative work with the western mosquitofish, G. affinis, provided further clues that this synaptic plasticity pathway is involved in mediating female responses during high and low sexual conflict encounters (Fig. 4c, d). Parallel whole brain expression experiments with female western mosquitofish exposed to conspecific (all coercive) males revealed a negative correlation between synaptic plasticity expression and preference behavior (Fig 4c, Lynch et al. 2012). In order to determine whether this response in G. affinis females represented a fixed difference between G. affinis and X. nigrensis or was a product of the type of males they each interact with, we performed a subsequent experiment exposing female G. affinis to alternative reproductive phenotypes of the heterospecific (sailfin mollies, P. latipinna). When exposing female G. affinis to either courting or coercive sailfin molly males, Wang et al. (2014) found both a positive and negative covariance pattern predicted by male phenotype (Fig. 4d). Finding the same negative correlation between synaptic plasticity expression in the brain and preference behavior when female G. affinis were interacting with both conspecific or heterospecific coercive types suggested that females are responding to a male reproductive tactic more so than species identity. Furthermore, the similarities between G. affinis and X. nigrensis females when interacting with courting males (a positive correlation between brain synaptic plasticity expression and preference behavior) suggest that female poeciliids may share a conserved neural response towards high and low sexual conflict encounters.

Further evidence that high and low sexual conflict encounters make differential demands on neural processing comes from comparative research within female X. nigrensis brains (Fig. 5). By examining the correlational patterns of the expression of learning genes (synaptic plasticity gene markers) across brain regions in the social decision-making network during different social encounters, we can get early clues as to which brain regions are involved. Our research revealed that social encounters with courting males evoked a more complex pattern of synaptic plasticity co-expression across the SDMN within female X. nigrensis brains than when females encountered only coercive types (see Fig. 5, Wong et al. 2012; Wong and Cummings 2014). Specifically, as females engage in social interactions with courting large male phenotypes (L exposed females in Fig. 5a, b), a greater number of brain regions associated with social decision making and learning are expressing genes that encode for rewiring, strengthening, and making novel neuronal connections than when females are engaged in social interactions involving only coercive males (Fig. 5c). This scaling pattern suggests that interactions with courting phenotypes, more so than coercive phenotypes, demand greater learning or memory engagement.

Fig. 5
figure 5

Co-expression of neuroligin-3 (a synaptic plasticity related gene that is linked to learning processes in the brain) across different brain regions in female X. nigrensis exposed to different sexual conflict environments. Individual female X. nigrensis were exposed to different social encounters for 30 min in a non-contact dichotomous choice experimental arena where conspecific stimuli were either two large courting males (LL, n = 10 female subjects), a large courter and a small coercer (LS, n = 13 female subjects), two small coercive males (SS, n = 7 female subjects), or two size-matched females (FF, n = 12 female subjects). Female behavior was scored (see Fig. 4b) and females immediately sacrificed after the experiment for brain gene expression analysis. We employed in situ hybridization of neuroligin-3 (redrawn with permission from Wong and Cummings 2014) and neuroserpin (not shown, Wong et al. 2012). In the schematic sagittal section of an X. nigrensis brain, each line represents unique significant (p < 0.05) positive correlations of gene expression between pairs of brain regions for each male social exposure group (LL, LS, SS) relative to the FF controls. Brain regions include many regions within the social decision-making network including the fish homologue to the mammalian: cerebellum (Cb), pallial hippocampus (Dl), basolateral amygdala (Dm), periaqueductal gray (GC), ventral hypothalamus (HV), pituitary (Pit), preoptic nucleus (POA), ventromedial hypothalamus (TA), medial amygdala (Vs), and the ventral nucleus of the ventral telencephalon (Vv)

Reconciling current evidence and identifying fruitful avenues for the future

At first blush, our whole brain (Fig. 4) and within brain (Fig. 5) neurogenomic research with swordtails and western mosquitofish might appear to paint a contrary picture to the artificial selection experiments conducted with guppies and eastern mosquitofish. The artificial selection experiments suggest an increasing demand for cognitive hardware with increasing sexual conflict, whereas our research suggests greater demand for learning and memory processes when females interact with lower conflict males (courters) over high-conflict males (coercers). Yet, these two distinct datasets may represent compatible and complementary snapshots of the coevolutionary process between sexual conflict and cognition in poeciliids.

Sexual coercion is generally assumed to be the ancestral mating system within the Poeciliidae family (Bisazza 1993; Ptacek and Travis 1998). Meanwhile, male courtship behavior is considered to be a derived behavioral trait (Bisazza 1993), and has been proposed to be an evolutionary outcome of high sexual conflict (Wang et al. 2015). Specifically, as sexual conflict escalates between males and females, one possible outcome is the evolution of an alternative male phenotype that de-escalates conflict by engaging in cooperative rather than coercive mating behavior (courtship; Wang et al. 2015). Courtship benefits females by lowering costs associated with excessive mating, and benefits males by increasing insemination efficiency (Pilastro and Bisazza 1999). A comparison of sexual conflict morphologies (gonopodial and gonopore features) and courtship display within 10 species in the Gambusia genus revealed a coevolutionary pattern with courtship behavior emerging as sexual conflict escalates (Wang et al. 2015). If this pattern is robust across Poeciliidae, then we expect to find patterns of increasing sexual conflict preceding the emergence of alternative mating tactics and phenotypes that rely on courtship. An additional outcome of this pattern would be that as sexual conflict increases, so does the complexity of the mating landscape, and this places a high demand on female cognitive processes (Cummings and Ramsey 2015).

In general, complex social interactions are predicted to place a higher demand on cognitive hardware than simple ones (Dunbar and Schultz 2007; Bshary et al. 2014; Cummings and Ramsey 2015; Matta et al. 2016). As the complexity of sexual conflict interactions increase, we expect higher cognitive demands (Buechel et al. 2016). In addition, when females have to navigate more than a single male reproductive phenotype or tactic, we expect the cognitive demands to be even higher (Cummings and Ramsey 2015). Combining the current empirical datasets with theoretical predictions, the following proposed scenario emerges. When male coercive tactics are simple and predictable, there is relatively low demand on female cognitive processes for counter behavioral responses. As males evolve more complex coercive tactics, this places a higher demand on female cognitive processes to circumnavigate pressures pushing her away from her optimal mating rate. Finally, as alternative male reproductive phenotypes (or tactics) evolve, this places even further demands on female cognition to discriminate amongst discrete male types and/or contexts that represent cooperative verses coercive mating opportunities. Further manipulative research directly comparing social complexity verses sexual conflict alone will enable us to test these emerging hypotheses.

As we continue to pursue the questions above with poeciliids, we continue to build a grander comparative view enabling us to illuminate the boundaries of the intertwined relationship between sexual conflict and cognition. Is the relationship between sexual coercion and cognition linear? Or are there levels of coercion and sexual conflict that promote a reduction in cognitive processes? How do cooperative sexual interactions stack up against sexual conflict as an evolutionary catalyst for cognitive enhancement? Such questions have no current answers, but there is much promise within the poeciliid system to find these answers in the near future.