Keywords

1 Overview

Motivation is a dominant construct in psychology, psychiatry, and neuroscience, as trying to understand why animals, including humans, do what they do is at the core of these disciplines. Although motivation can be defined in a variety of ways, a key component is that motivated behaviors are directed toward (approach) or away (avoidance) from a stimulus. Motivation also contains emotional elements with approach linked to positive hedonic valence and avoidance linked to negative valence. This review focuses on social motivation, which, like other forms of motivation, is influenced by past experience and an individual’s internal state. Social motivation is, however, intrinsically more dynamic and less predictable because the drive to approach or avoid another individual(s) depends in large measure on how that individual behaves.

Recent studies of the neurobiology of social behavior have often characterized social behavior as having a positive valence, described as prosocial or affiliative interactions (e.g.,pair bonding , maternal behavior), or as having a negative valence, described as negative social interactions (e.g., aggression, territoriality). Although such a dichotomy is convenient and can have descriptive value, a closer look at these behaviors suggests that social motivation is more complex. For example, while the formation of a pair bond in a species like prairie voles has positive behavioral elements, such as highly affiliative behaviors directed toward a partner, it is also associated with mate guarding, in which males display selective aggression toward other voles. Thus, in the context of a pair bond, simply ascribing positive valence to the affiliative behaviors and negative valence to aggression is an oversimplification. Further, all aggressive behaviors are not the same, nor are the effects on the players. The fact is, winning is rewarding (Martinez et al. 1995; Meisel and Joppa 1994), and there is even the possibility that losing can be rewarding as long as the defeat is not too severe (Gil et al. 2013). Therefore, assigning hedonic valence to social behaviors (e.g., aggressive behavior) or to mating strategies (e.g., pair bonding) must be done with great care, particularly when linking approach or avoidance with the neural mechanisms underlying motivation.

Historically, investigations into the neurobiology of motivation have primarily focused on the mesolimbic dopamine (DA) system where DA neurons were thought of as “reward” neurons. It has now been recognized that the role of the mesolimbic DA system in hedonic mechanisms is far more complex. Understanding social motivation requires us to expand our studies of the neural mechanisms of motivation beyond this system into the networks that control the expression of social behavior in response to social stimuli. Of the myriad of neurochemical signals that are known to be involved in the modulation of social behaviors, two neuropeptides, oxytocin (Oxt) and vasopressin (Avp), stand out as being critical across species. Because of the vast literature on the role of these two nine-amino acid neuropeptides, or nonapeptides, in regulating social behavior, this chapter will focus on mammalian social behavior and provide examples of the powerful contributions of these two neuropeptide systems to cooperative and competitive behaviors.

2 Origins and Mechanisms of Motivated Social Behaviors

When considering the evolutionary origins of motivated behaviors, most simply put, it all comes down to fitness. Animals engage in species-specific behaviors because over evolutionary time these behaviors were either selected for via natural or sexual selection, or occurred through some other mechanism of evolution. In the context of sexual behavior, males are described as “ardent” and females as “choosy,” which is reflected in their physiology. Males make a lot of sperm and often display behaviors that that will result in the fertilization of as many eggs as possible over their reproductive lifetime. Female mammals on the other hand typically have to invest in the gestation and the care of offspring, so they tend to be more selective about their mates. These differences in selective pressures result in vastly different behavioral displays between males and females. However, these behavioral differences are not limited to sexual behaviors. For instance, in mammals, biparental behavior is scarce, occurring in fewer than 6 % of rodent species and 5–10 % of all mammals (Kleiman 1977). Thus, a male that engages in parental care does so because the “cost” of not being paternal is too high; for instance, while reproductive success may be compromised, his proximity to the female increases the likelihood that he is the sire of the offspring. There are also sex differences in displays of cooperative and competitive behaviors, with females often displaying more cooperative behaviors across the lifetime and males displaying more competitive behaviors, particularly during the breeding season.

In nature, the diversity in cooperativity and competitiveness observed across animal species is striking. Some animals have social structures that are characterized by high levels of cooperativity, such as that observed in species that form long-term social bonds like pair bonds . In other species, high levels of competitive behaviors serve to establish and maintain social dominance relationships. Overlaid on the complexity of social life for a given species is a lack of stability, as social behaviors often change over the seasons and over the lifetime. Further, the different behavioral strategies employed by an animal have their particular costs and benefits. To explore these costs and benefits we can take a closer look at cooperative and competitive behaviors.

2.1 Cooperative and Competitive Behaviors

Cooperative behaviors, associated with affiliative behaviors, are thought to have evolved from reproductive and parental behaviors, in turn being permissive for the development of longer-term social bonds (Crews 1997). Competitive behaviors too are important in the formation of social bonds, as intraspecific interactions are universal and often governed by dominance relationships. Some evolutionary advantages to forming social bonds include localization of resources, lower predation due to group aggression , and increased reproductive opportunities (Alexander 1974). Social bonds have been extensively studied in primates and in some instances have been shown to increase evolutionary fitness (Silk 2007). In free-ranging baboons, females that have strong social bonds with one another live longer than those who have weaker social bonds (Silk et al. 2009). Even in humans social relationships can have profound effects on an individual’s health, including improved mood and a longer life (House et al. 1988; Rodriguez-Laso et al. 2007; Baumeister and Leary 1995).

However, being social does have its cost, such as increased susceptibility to disease, parasites, or injury (Alexander 1974; Crews 1997). In order for animals to live in groups, they must be able to tolerate close proximity; thus, keeping levels of aggression in check becomes particularly important. It also requires a memory of the members of the social group, as this allows animals to identify familiar stimuli, which in turn is permissive for adaptive behavioral responses. While many mammalian species live in groups, some species show “shifts” in the nature of their social interactions depending on where they are in their breeding cycle. Some species display high levels of affiliative behaviors in the non-breeding season, while others increase their intraspecific aggression when resources are scarce (Anacker and Beery 2013).

Over the last several decades, investigation of the neural mechanisms underlying social behavior has focused primarily on “prosocial” behaviors, rather than competitive behaviors. The tremendous progress that has been made in understanding the neural mechanisms underlying phenomena such as maternal behavior and pair bonding has likely contributed to this imbalance. Unfortunately, investigation of more competitive behaviors, such as aggression, has been on the decline for a variety of reasons (see Blanchard et al. 2003). Within studies of competitive behavior, males have been the main experimental subjects, perhaps because of Darwin’s emphasis on male–male competition and female mate choice in the context of sexual selection (Darwin 1871). More recently, however, the importance of competitive behaviors in females has been recognized. Not only do female mammals compete for resources and mates to achieve reproductive benefits, but female competition is widespread in the animal kingdom (Rosvall 2011; Stockley and Bro-Jorgensen 2011; Huchard and Cowlishaw 2011). Females compete for resources such as food, nest sites, and protection using a variety of strategies including intergroup aggression, dominance relationships, and territoriality, as well as through the inhibition of the reproductive capacity of other females. In many primate species female aggression is associated with rank and ultimately reproductive goals (for review, see Stanyon and Bigoni 2014). As a result, investigation of female competitive behavior is essential to understanding social behavior and its translational implications.

In rodents , one reason that there are few data on female competitive behavior is that in commonly studied laboratory species, i.e., rats and mice, females display little or no competitive behavior (Blanchard and Blanchard 2003). This contrasts with what is observed in another laboratory rodent, whose utility as an experimental model continues to increase, Syrian hamsters (Mesoscricetus auratus). Female Syrian hamsters display a range of competitive strategies including the expression of high levels of spontaneous offensive aggression, the rapid formation of robust dominance relationships, and the ability to inhibit the reproductive capacity of other females (Albers et al. 2002; Huck et al. 1988). While there is very little known about the neural mechanisms controlling female offensive aggression in any mammalian species, studies in hamsters have provided a good deal of information about how gonadal hormones influence this form of aggression (Albers et al. 2002).

In males, high levels of aggressive behaviors, intermale and territorial in particular, are often at their peak with the onset of breeding. Other forms of aggression are closely linked to parental behaviors, thus allowing for the defense of young, mates, food, or territories. Seasonal shifts from high cooperativity/affiliative behaviors to competitive behaviors are mediated primarily by changes in gonadal steroids, which are often linked to changes in photoperiod, though numerous neurotransmitter/neuropeptides are also involved. In many mammalian species, androgen concentrations are very high during the breeding season, as they are needed to support reproductive behaviors as well as the physiology of the gonads. During the non-breeding season, seasonal breeders will often undergo a period of gonadal quiescence, whereby the testes shrink in size and levels of circulating androgens plummet. However, it should be noted that in some species, such as hamsters (also seen in birds), androgens, specifically dehydroepiandrosterone (DHEA), produced by the adrenal glands may help to support aggressive behavior during the non-breeding season by serving as a prohormone or neurosteroid for the brain when gonadally derived androgen levels are low (for review, see Soma et al. 2015).

2.2 Brain Areas that Regulate Cooperative and Competitive Behaviors

The social behavior neural network (SBNN) hypothesis by Newman (1999) proposes that a network composed of neural groups or “nodes” including, but not limited to, the extended amygdala, the bed nucleus of the stria terminalis (BNST), lateral septum (LS), periaqueductal gray (PAG), medial preoptic area (MPOA), ventromedial hypothalamus (VMH), and anterior hypothalamus (AH) controls social behavior. Each node within the SBNN meets several criteria: reciprocal connectivity, neurons with gonadal steroid hormone receptors, and having been identified as being important to more than one social behavior. The SBNN hypothesis has gained traction in the field in recent years (reviewed by Albers 2012, 2015; Crews 1997; Goodson and Kingsbury 2013). It represents a more nuanced and complicated approach to the understanding of social behavior, as it takes the regulation of these behaviors beyond the examination of just a single neuroanatomical area and supposes that the output of the network is an emergent property. The identification of SBNN for different species is an important next step in understanding the complexity of behavior. While previous approaches have been more simplistic, examining specific neural anatomical areas, single neurotransmitter/neuropeptides, and behaviors, the foundation has now been laid for impactful studies focused on how social behavior emerges from complex neural networks. A large number of different types of motivated social behaviors are thought to be controlled by the SBNN, including offensive and defensive aggression , social recognition memory , parental behavior, and social communication. Importantly, Oxt/Avp and their receptors are found throughout the SBNN and are ideally suited to regulate the expression of social behavior because of their plasticity in response to factors that influence social behavior (Fig. 1) (reviewed in Kelly and Goodson 2014; Goodson and Kingsbury 2013; Albers 2012, 2015; Caldwell et al. 2008a; Adkins-Regan 2009; Bosch and Neumann 2012).

Fig. 1
figure 1

Oxytocin and vasopressin signaling in brain areas important to social motivation. Oxytocin (Oxtr) and vasopressin receptors (Avprs) are found throughout the structures of the social behavior neural network (SBNN) and the mesocorticolimbic dopamine (DA) system. Their localization in these nuclei is critical for oxytocin’s and vasopressin’s modulation of socially motivated behaviors and may serve as the functional connection between the SBNN and DA systems, particularly by their action in the lateral septum (LS) and extended amygdala, including the bed nucleus of the stria terminalis (BNST). Other abbreviations: AH anterior hypothalamus; BLA basolateral amygdala; HIPP hippocampus; MeA medial amygdala; NAcc nucleus accumbens; OB olfactory bulb; PAG periaqueductal gray; PFC prefrontal cortex; POA preoptic area; VP ventral pallidum; VTA ventral tegmental area; VMH ventromedial hypothalamus

Full size image

Motivated behaviors also arise from a network of reciprocally connected brain regions that determine the salience of stimuli, assign motivational value, and initiate appropriate action (reviewed by Love 2014). The ventral tegmental area (VTA) is a key region in this network in that VTA neurons producing DA project to a large number of cortical and limbic structures, forming the foundation underlying the motivational circuitry. This network plays a critical role in social as well as nonsocial behavior and appears to provide an alerting signal for unexpected stimuli. Within this network, there are distinct groups of DA neurons that determine motivational value, being excited by appetitive stimuli and inhibited by aversive stimuli. Other groups of DA neurons appear to encode motivational salience, but not valence, in that they are excited by the intensity of the stimulus, regardless of whether it is appetitive or aversive (reviewed by Love 2014). See chapters by Bissonette and Roesch, Robinson et al., Redish et al., and Salamone et al. for detailed discussions of these issues.

While the SBNN and the mesolimbic DA system are distinct from one another, they are thought to dynamically interact and support decision making in the context of motivated social behaviors (O’Connell and Hofmann 2011a, b). O’Connell and Hofmann (2011b) have proposed that these two systems should be considered as part of a larger social decision-making network (SDM) that is relatively conserved across species. Across these two systems, there are two neuroanatomical areas, or nodes, of overlap—the LS and the extended amygdala—including the BNST. These areas provide the functional connection between the two systems by acting as relays, providing the SBNN with information from the motivational network about the salience of a social stimulus in turn allowing for an appropriate behavioral response. With these concepts in mind, in the following sections, we will discuss the possibility that Oxt and Avp provide critical links between specific elements within the SBNN and the motivational network that contribute to the motivational forces driving social behaviors.

3 Neuroendocrine Modulation of Social Behaviors

The first hormones implicated in the regulation of social behaviors were the gonadal steroids (Berthold 1849), since changes in social behaviors are observed following gonadectomy. While the gonadal steroids are important, so too are the evolutionarily ancient Oxt and Avp neuropeptide systems, as well as their non-mammalian homologues. Oxt and Avp are both primarily synthesized in the paraventricular nucleus (PVN) and supraoptic nucleus (SON) of the hypothalamus. Their genes sit in opposite transcriptional direction on the chromosome as the result of the duplication of an ancestral vasotocin gene (Acher and Chauvet 1995; Acher et al. 1995) and they are synthesized as part of a larger precursor preprohormone (Hara et al. 1990). Since they are so structurally similar, Oxt and Avp are considered “sister” hormones, though their actions both peripherally and centrally can differ significantly from one another. Interestingly, across mammalian species, the roles of Oxt and Avp with regard to cooperative and competitive behaviors tend to be fairly conserved (Caldwell et al. 2008a; Caldwell and Young 3rd 2006; Lee et al. 2009a; Adkins-Regan 2009; Neumann 2008; Veenema and Neumann 2008; Carter et al. 2008; Albers 2012, 2015).

3.1 The Oxytocin System

Oxytocin literally meaning “sharp childbirth” is known for its peripheral actions on the regulation of uterine contraction as well as the facilitation of milk ejection (Dale 1906; Ott and Scott 1910). In rats, Oxt is synthesized in larger, magnocellular neurons, of the PVN and SON that project to the posterior pituitary and mediate the aforementioned actions. However, it is Oxt that is synthesized in the smaller, parvocellular neurons of the PVN that project centrally and mediate many of the central actions of Oxt. It should be noted, however, that this compartmentalization of function by magnocellular versus parvocellular neurons is not found in all species. For example, in Syrian hamsters, magnocellular neurons do not exclusively project to the posterior pituitary and seem to project centrally (Ferris et al. 1992b). Also, in mice and several vole species, there are reports of parvocellular Oxt neurons outside of the PVN (Castel and Morris 1988; Jirikowski et al. 1990; Wang et al. 1996) as well as reports of Oxt collaterals from the SON/PVN to the nucleus accumbens (NAcc) (Ross et al. 2009a).

Thus far, only a single Oxt receptor (Oxtr) has been identified, and it is thought to be the primary mechanism for the transduction of the Oxt signal (Kimura et al. 1992; Kubota et al. 1996); however, see the section titled “Signaling by Oxytocin and Vasopressin in the Brain.” The Oxtr is a member of the seven-transmembrane G-protein-coupled receptor family and signals through Gαq/11 GTP-binding proteins and Gβλ (Ku et al. 1995; Gimpl and Fahrenholz 2001; Zingg and Laporte 2003), which results in the hydrolysis of phosphatidylinositol. The structure and sequence of the Oxtr is similar to the Avp receptors (Gimpl and Fahrenholz 2001). In rats and mice, the Oxtr is most often visualized with receptor autoradiography through the use of a potent and specific 125I-labeled antagonist (Kremarik et al. 1993; Veinante and Freund-Mercier 1997). The Oxtr is observed in several areas of the brain, including the hippocampal formation, LS, central amygdala (CeA), olfactory tubercle, NAcc shell, dorsal caudate–putamen, BNST, medial amygdala (MeA), and VMH (Kremarik et al. 1993; Veinante and Freund-Mercier 1997; Insel et al. 1991), but there are seasonal as well as species- and sex-specific differences.

3.2 The Vasopressin System

Avp is named for its involvement in the constriction of blood vessels, but is also important to salt and water balance. The peripheral actions of Avp are primarily mediated by the magnocellular neurons of the PVN and SON, which result in Avp release from the posterior pituitary. Centrally, Avp is more widely expressed than Oxt, and its distribution can vary substantially between species. Avp immunoreactive (ir) cell bodies are consistently found in several hypothalamic nuclei including the suprachiasmatic nucleus (SCN), PVN, SON as well as in groups of accessary nuclei (Sofroniew 1983). Outside of the hypothalamus, Avp-ir neuronal cell bodies can be observed in the BNST and MeA in most rodent species examined to date (Sofroniew 1985). Interestingly, in Syrian hamsters, neuronal cell bodies containing Avp are absent in the BNST and MeA (Albers et al. 1991). Projections from Avp-producing neurons form a dense vasopressinergic network throughout the brain (Buijs et al. 1983, 1987; De Vries and Buijs 1983; Sawchenko and Swanson 1982).

Avp receptors can be divided into two classes: Avp1 and Avp2 receptors (Avpr1 and Avpr2, respectively), both of which are seven-transmembrane G-protein-coupled receptors that are similar in structure to the Oxtr. There are two subtypes of the Avpr1: the Avpr1a and the Avpr1b. Peripherally, the Avpr1a mediates the effects of Avp on vasoconstriction and can be found in the liver, kidney, platelets, and smooth muscle (Ostrowski et al. 1992; Watters et al. 1998). Centrally, the Avpr1a is found in a variety of brain nuclei (Johnson et al. 1995; Tribollet et al. 1997; Ostrowski et al. 1994; Szot et al. 1994). The Avpr1a is modulated by gonadal hormones and photoperiod in some brain regions, but not others (Johnson et al. 1995; Young et al. 2000; Caldwell and Albers 2004b; Caldwell et al. 2008b). The Avpr1b was originally described in the anterior pituitary, where it is prominent on the corticotrophs; however, it can also be found in the brain (Antoni 1984; Lolait et al. 1995). In rats, there is a lack of consensus about the central distribution of the Avpr1b, with some groups reporting Avpr1b in the olfactory bulb, piriform cortical layer II, LS, cerebral cortex, hippocampus, PVN, SCN, cerebellum, and red nucleus (Lolait et al. 1995; Saito et al. 1995; Vaccari et al. 1998; Hernando et al. 2001; Stemmelin et al. 2005). However, a later study by Young and colleagues, which used more stringent conditions for in situ hybridization histochemistry (ISHH), determined that the Avpr1b in rats and mice is more discretely localized with prominent expression in hippocampal field CA2 pyramidal neurons (Young et al. 2006). The Avpr2 is found in the periphery and is primarily expressed in the kidney; it has not been localized to the brain. Its role in the kidney is to transduce the antidiuretic effects of Avp within the renal collecting ducts (Bankir 2001).

3.3 Signaling by Oxytocin and Vasopressin in the Brain

Neuropeptides can act in a highly localized manner, similar to classic neurotransmitter release at the synapse. However, neuropeptides can also be released in a much more diffuse manner, potentially impacting large numbers of neurons at multiple sites (Engelmann et al. 2000; Landgraf and Neumann 2004; Ludwig 1998; Ludwig and Leng 2006). This diversity of action has long been recognized, although the role of these different types of signaling mechanisms in the control of social behavior is not well understood. Neuropeptides such as Oxt and Avp are usually packaged in large dense-core vesicles (LDCV) that can be found in all areas of neurons including the presynaptic terminal (Jakab et al. 1991; Buijs and Swaab 1979; van Leeuwen et al. 1978). Because of the broad distribution of LDCV throughout the cell, neuropeptides can act locally at the synapse or much more broadly when released from non-synaptic regions (e.g., dendrites) in what is called volume transmission. Many factors affect the profiles of neuropeptide release such as the size of the neurons from which they are released, the spread of peptide after release, and the timing and intensity of its degradation by peptidases. While the spatial and temporal profiles of peptide release via volume transmission are not well understood (Leng and Ludwig 2008), estimates suggest that they may travel as far as 4–5 mm from their site of release (Engelmann et al. 2000). Magnocellular neurons in the hypothalamus represent the largest pool of nonapeptides in the brain, and there is evidence that they are activated by a variety of stimuli related to social behavior (Delville et al. 2000). As a result, it seems likely that volume transmission of nonapeptides from these neurons plays an important role in regulating social behavior by acting on nonapeptide receptors throughout the SBNN .

Another consideration is that neuropeptides are commonly found in neurons that also produce small molecular weight “classical neurotransmitters” (e.g., amino acids) (for review, see van den Pol 2012; Albers 2015), thus allowing for their co-release. In most cases, classical neurotransmitters are packaged in small synaptic vesicles (SSVs) in presynaptic terminals. The exocytosis of both SSVs and LDCVs at synapses is Ca2+ dependent. Because SSVs are usually in closer proximity to the membrane than LDCV, less activity is required for classic neurotransmitter release than for neuropeptide release. Therefore, synaptic release of neuropeptides is thought to lag behind that of neurotransmitter release and to require more electrical activity. The functional significance of synaptic co-release of classical neurotransmitters and neuropeptides is not known, but dynamic interactions of these signals will likely be important in understanding neuropeptide regulation of behavior (Bamshad et al. 1996). In summary, the ways that Oxt and Avp contribute to neurochemical signaling within the brain are varied and complex. As a result, researchers are left to untangle very intricate pharmacological interactions when they consider the effects of these neuropeptides on behavior.

In addition to the diversity seen in the ways that Oxt and Avp can signal, Oxt and Avp also have a high degree of similarity in their structure as well as in the structure of their canonical receptors (Maybauer et al. 2008; Gimpl and Fahrenholz 2001; Manning et al. 2012; Song et al. 2014b). Due to these structural similarities, there is a substantial amount of cross talk between these systems (Schorscher-Petcu et al. 2010; Sala et al. 2011), with Oxt and Avp having similar affinities for the Oxtr, Avpr1a, and Avpr1b in rats and mice (Manning et al. 2008, 2012). For example, both Oxt and Avp can induce communicative behavior in hamsters when injected into the AH by activating Avpr1a rather than the Oxtr (Song et al. 2014b). Similarly, both Oxt and Avp can enhance social recognition and social reward by activating the Oxtr and not the Avpr1a (Song et al. 2014a, b, 2015; Ragnauth et al. 2004). These data indicate that the effects of Oxt and Avp on different social behaviors can result from their activation of the Avpr1a, the Oxtr, or both.

Oxt and Avp receptors are distributed within structures of the mesolimbic DA system, providing a means by which these systems may interact. These areas include the amygdala, the hippocampus, the VTA, the prefrontal cortex (PFC), the NAcc, and the ventral pallidum (VP) (Vaccari et al. 1998; Baskerville and Douglas 2010; Curtis et al. 2008). It is also important to remember that there is a great deal of interspecies and interindividual variability in the distribution of Oxt and Avp receptors and that these differences play a major role in producing differences in the expression of social behavior.

Recent work in primates has shed some light on the complexity of these systems in a broader evolutionary context, as there are some significant evolutionary changes in the Oxt and Avp systems. As discussed above, in rats and mice, the Oxtr and Avprs are relatively non-selective for Oxt and Avp; however, in humans, the OXTR has significant selectivity for Oxt over Avp. The functional significance of these differences in receptor selectivity is not known but is likely to be important. Aside from cross talk, there are also sequence differences in Oxt across primate species. Specifically, work by Lee et al. (2011) challenged the idea that Oxt is an invariant nine-amino acid sequence by determining that some New World primates have an amino acid substitution in the 8th position, a leucine rather than a proline. The implications of these findings are still being explored (Cavanaugh et al. 2014), but this coupled with variations in the Avpr1a gene suggests that there has been, and perhaps continues to be, an evolutionary shift in these systems and reinforces the importance of studying these systems together rather than in isolation (for review, see Ragen and Bales 2013). These findings are likely to impact work in preclinical models and may help to resolve some of the conflicting findings in the literature between primates and other mammalian species.

3.4 Epigenetics

The advent of epigenetics has also challenged our understanding of how these systems are regulated and potentially how they regulate other systems. Epigenetics refers to changes in gene transcription that are not due to changes in nucleotide sequence, but rather are “above” the genome. Epigenetic mechanisms include histone modification and gene methylation, both of which alter the ability of the transcriptional machinery to access promoter regions. While much of this work originally centered on Oxt and maternal behavior in animal models, it has now been expanded to human studies.

Elegant work from Dr. Michael Meaney’s laboratory found that in rats, the quality of mother–infant interactions affected DNA methylation and histone acetylation patterns in the offspring (e.g., Champagne et al. 2001, 2004; Fish et al. 2004; Francis et al. 1999; Weaver et al. 2004). During the first postpartum week, low licking-and-grooming (LG) dams have reduced Oxtr binding in the MPOA compared to high LG dams (Champagne et al. 2001; Francis et al. 2000). Further, microinjection of an Oxtr antagonist reduces licking and grooming in high LG dams, with essentially no effect in low LG dams (Champagne et al. 2001). For more complete reviews of this work, see (Champagne 2008; Bridges 2015).

More recently, work in humans suggests that changes in the methylation of the OXTR are associated with a variety of diseases/disorders, including anorexia nervosa (Kim et al. 2014), the perception of fear and anxiety (Puglia et al. 2015; Ziegler et al. 2015), psychopathy (Dadds et al. 2014), and autism (Gregory et al. 2009). However, a direct causal link between methylation patterns and behavior has yet to be made; however, given the animal literature, it seems likely that this is an important means by which early life experience may be able to directly impact behavior.

4 Oxytocin/Vasopressin and Cooperative and Competitive Behaviors: Social Memory, Social Interactions, and Aggression

Within mammals, social structures can vary widely. Take for instance naked mole rats and their large insect-like colony structure, prairie voles and their pair bonding that results in a lifetime social dyad, or more solitary species such as Syrian hamsters. With this diversity, it might be assumed that there are vast differences in the types of behaviors that these species are capable of displaying and in the neurotransmitters and/or brain areas that are important to their regulation. This, however, does not seem to be the case. While the details may differ, there are a limited number of social behaviors common to most species and they include the ability to remember others of the same species (social memory ) as well as the social behaviors that determine the nature of their relationships with conspecifics (e.g., affiliation/dominance). Furthermore, the roles of Oxt and Avp in the modulation of social behaviors have been evolutionarily conserved across species. For example, aggression is modulated by social experience-induced changes in the expression of Avpr1a in the hypothalamus in both male prairie voles and hamsters.

Since 90 % of mammals are not biparental, it is common for them to live in large social groups and display both cooperativity and competitiveness, depending on the context. Even in species that are more solitary and highly competitive, individuals have the capacity for social recognition, social communication, as well as the potential to form stable long-standing social relationships. There are also sex differences as well as environmental effects, such as photoperiod, which can cause shifts in social behaviors from the breeding to the non-breading season. In this section, we will explore the role of Oxt and Avp in selected forms of cooperative and competitive behavior in these differing and complex social contexts.

4.1 Social Recognition Memory

Displays of social behaviors often depend on whether the interaction is with an individual that is familiar or unknown. Thus, the ability to recognize individuals and remember them, i.e., social recognition memory, plays an important role in the decision to approach or avoid. There are considerable data that both Oxt and Avp contribute to social recognition memory. Oxt is thought to facilitate social memory by altering the processing of socially salient olfactory information (for review, see Lee et al. 2009a; Gabor et al. 2012; Wacker and Ludwig 2012). In males, Oxt infused into the olfactory bulb (OB), lateral ventricles, and MPOA facilitates social recognition memory (Dluzen et al. 1998; Benelli et al. 1995; Popik and Van Ree 1991). Some of the particulars of the circuit have been worked out with the assistance of genetic knockouts of the Oxt system, with Oxt knockout (Oxtr −/−) mice and forebrain-specific Oxtr knockout (Oxtr FB/FB, where CRE recombinase is driven by a CaMKIIα promoter) mice having deficits in social recognition memory (Lee et al. 2008a, b; Takayanagi et al. 2005; Macbeth et al. 2009; Ferguson et al. 2000; Hattori et al. 2015). Based on this work, a four-gene micronet involving Oxt, the Oxtr, estrogen receptor α, and estrogen receptor β has been proposed as being critical to the regulation of social recognition memory in both males and females (Choleris et al. 2003). In particular, estrogen-dependent Oxt signaling in the MeA appears to be key for normal social recognition memory. Infusion of Oxt into the MeA of Oxt −/− mice can rescue deficits in social recognition memory (Ferguson et al. 2001), and infusion of an Oxtr antisense DNA or an Oxtr antagonist into control mice can block social recognition memory (Choleris et al. 2007; Ferguson et al. 2001). Female Oxt −/− mice also have a disrupted Bruce effect (Bruce 1959), whereby they terminate their pregnancy when exposed to their mate, which suggests that they do not remember him (Wersinger et al. 2008).

Recent work suggests that Oxt may also play a role in human social recognition. Generally speaking, exogenous Oxt enhances the memory for faces (Savaskan et al. 2008; Guastella et al. 2008; Rimmele et al. 2009). Further, a common single-nucleotide polymorphism (SNP) (rs237887) in the OXTR is moderately associated with face recognition memory in families from the UK and Finland that have a child with an autism spectrum disorder (ASD) (Skuse et al. 2014). Taken together, these data suggest that Oxt and the Oxtr have a conserved role in the modulation of social memory across species.

Avp also appears to be important for social memory. Androgen-dependent Avp projections from the MeA and BNST to the LS, all parts of the SBNN , are important for individual recognition (De Vries et al. 1984; Mayes et al. 1988; Bluthe et al. 1990, 1993). Microinjections of Avp into the LS of control or AVP-deficient Brattleboro rats facilitates social memory, whereas microinjections of Avpr1a antagonists or infusions of antisense Avpr1a oligonucleotides into the LS of control rats impairs social recognition memory (Engelmann and Landgraf 1994; Landgraf et al. 1995). The use of Avpr1a −/− and Avpr1b −/− mice has also provided some insight into the contributions of Avp to social memory. However, in the NIMH line of Avpr1a −/− mice, the findings have been mixed (Hu et al. 2003), with one group reporting that males have impaired social recognition that can be rescued by the overexpression of Avpr1a in the LS (Bielsky et al. 2003, 2005; Bielsky and Young 2004) and another group reporting no deficits in social recognition, but rather in olfaction (Wersinger et al. 2007b). While the reason for the discrepancy remains unknown, it is obvious from previous reports that Avpr1a in the LS is important for normal social recognition memory. Interestingly, Avpr1b −/− mice have mild impairments in social recognition memory (Wersinger et al. 2002). Further, lesions and genetic silencing of the CA2 region of the hippocampus, where the Avpr1b is prominently expressed, also results in impaired social recognition memory (Stevenson and Caldwell 2012; Hitti and Siegelbaum 2014).

While the aforementioned data provide strong support for a role of Avp in social recognition, how or if Avp modulates social recognition has really only been studied in a small number of species. Further, the data are limited by the fact that the vast majority of studies have only examined social recognition for very short intervals (<2 h). However, there are some studies that suggest that Avp may be important for long-term social recognition. Specifically, injection of an Avpr1a antagonist into the LS can block the formation of a mating-induced pair bond in male prairie voles, while injection of Avp into the LS institutes a pair bond in the absence of mating (Liu et al. 2001). Since social recognition is a necessary part of pair bonding and pair bonds can last a lifetime, these data suggest that Avp signaling via the Avpr1a can induce long-term changes in social recognition. There are also data to suggest that Avpr1a antagonists administered into the MeA can alter maternal memories in rats. Normally, after a ten-day separation from their pups, mothers display full maternal behavior within about 12 h of re-exposure. However, in peripartum mothers, in which an Avpr1a antagonist is infused into the MeA, the latency to display full maternal behavior does not occur for approximately 60 h; the antagonist had no effect on the initial expression of maternal behavior on the day of parturition (Nephew and Bridges 2008).

More recently, we have found that male Syrian hamsters can recognize social odors of other male Syrian hamsters for 24 h. Injection of Oxt or Avp intracerebroventricular (i.c.v) extends their social memory to 48 h. Interestingly, these effects are mediated by the Oxtr and not the Avpr1a (Song et al. 2015). In the broader context of animal behavior, it will also be important to determine whether Avp mediates some of the more complex forms of social recognition found by rodents such as kin recognition and the true recognition of specific individuals (e.g., Mateo and Johnston 2003; Johnston and Peng 2008; Petrulis 2009).

4.2 Cooperative Behavior

Although cooperative behavior is certainly not limited to species that pair bond, pair bonding species do provide an important model system in which to investigate the neurobiology of cooperation. That said, it remains important to recognize that these bonds are formed by mating and are limited to the cooperation of a male and a female. Pair bonding, formed by mating, represents one form of cooperative behavior. Pair bonds are somewhat unique among mammals in that they are only seen in 3–5 % of mammalian species (Kleiman 1977). Defined as a preference for contact with a familiar sexual partner, selective aggression toward unfamiliar conspecifics, biparental care, socially regulated reproduction, and incest avoidance (Carter et al. 1995; Carter and Getz 1993), pair bonding can be found in titi monkeys (Callicebus cupreus), marmosets (Callithrix penicillata and Callithrix jacchus jacchus), California mice (Peromyscus californicus), and prairie voles (Microtus ochrogaster). Humans too can have strong selective bonds between mates, but also show cooperative behaviors in many other contexts as well. The extent to which human pair bonds and other forms of human cooperative behavior are regulated by Oxt and Avp remains to be determined.

Our understanding of the mechanisms by which Oxt and Avp contribute to pair bonding comes primarily from work in prairie voles. Prairie voles live in extended family groups and are considered a socially monogamous species (Carter et al. 1995) (social monogamy is distinct from sexual monogamy as most individuals have extra-pair copulations). The formation of a pair bond is experimentally tested in the laboratory using a partner preference test (Williams et al. 1992), whereby the preference of a male or female for an animal in which they have previously cohabitated, i.e., the partner, versus a “stranger,” is assessed. If the experimental subject spends twice as much time with the “partner” animal, then it is said to have formed a pair bond with that individual (Insel and Hulihan 1995; Carter and Getz 1993; Williams et al. 1994; Carter et al. 1995).

Due to the diversity in social structures within the genus Microtus, comparative studies between vole species have provided significant insight into the neural regulation of social bonding. By comparing the neurochemistry of monogamous vole species, such as the prairie or pine vole (Microtus pinetorum), to non-monogamous voles, such as the montane (Microtus montanus) or meadow (Microtus pennsylvanicus) voles, scientists have explored how variations in Oxt and Avp neurochemistry between highly related species can result in significant differences in social behavior (Young et al. 2008, 2011; Adkins-Regan 2009). While there are differences in Oxt- and Avp-ir cells or their projections between species, most profound are the changes in the neuroanatomical distribution of the Oxtr and the Avpr1a. Relative to non-monogamous voles, monogamous voles have higher densities of Oxtr, as measured using Oxtr autoradiography and ISHH, in the NAcc, the PFC, and the BNST. In contrast, promiscuous voles have higher Oxtr density in the LS, VMH, and the cortical nucleus of the amygdala (Insel and Shapiro 1992; Young et al. 1996; Smeltzer et al. 2006). Evidence that the differences in the distribution of the Oxtr between species might be behaviorally meaningful comes primarily from pharmacological studies.

In female prairie voles, central infusion of an Oxtr antagonist blocks the formation of the pair bond, but has no effect on sexual behavior, whereas central infusion of Oxt facilitates the pair bond in the absence of mating (Insel et al. 1995; Williams et al. 1994; Cho et al. 1999) and can decrease male-directed aggression (Bales and Carter 2003). In the aforementioned studies, the infusions were i.c.v.; however, manipulation of Oxtr signaling, using Oxtr antagonists and RNAi knockdown of the Oxtr within the NAcc, inhibits formation of a partner preference (Liu and Wang 2003; Young et al. 2001; Keebaugh et al. 2015), and overexpression of the Oxtr in the NAcc of adult female prairie voles accelerates the formation of partner preference (Ross et al. 2009b). However, overexpression of the Oxtr in the NAcc of non-monogamous meadow voles is not sufficient to promote pair bond formation (Ross et al. 2009b), which suggests that all of the required neurocircuitry is not in place for this species.

There are also differences in the distribution of the Avpr1a between vole species. Prairie voles have a higher density of Avpr1a, as measured using receptor autoradiography and ISHH, within the MeA, accessory olfactory bulb, diagonal band, thalamus, VP, and BNST compared to montane voles (Young et al. 1997; Insel et al. 1994). Montane voles, on the other hand, have a higher density of Avpr1a in the medial PFC and the LS (Smeltzer et al. 2006; Insel et al. 1994). These differences in Avpr1a distribution are thought to contribute to differences in social organization between monogamous and non-monogamous vole species. This hypothesis is supported by data in pine voles and meadow voles, which suggest similar social structure-specific distributions of Avpr1a between these species (Insel et al. 1994). Further support for this hypothesis comes from pharmacological manipulations of the Avpr1a in prairie voles. When an Avp antagonist is injected i.c.v. prior to mating, the formation of a partner preference is inhibited. Conversely, Avp infusion facilitates the formation of the partner preference (Winslow et al. 1993; Cho et al. 1999). Some of the more interesting data that support a role for the differential distribution of the Avpr1a in the formation of social bonds come from a study in which the prairie vole Avpr1a gene was overexpressed in the ventral forebrain of meadow voles, resulting in increases in the amount of time meadow voles spent huddled with their partners compared to controls (Lim et al. 2004).

4.3 Competitive Behavior

The most conspicuous form of competitive behavior is aggression . Offensive and defensive aggressions have been studied intensely and almost exclusively in male mammals. Further, neural circuits that overlap much of the SBNN have been proposed for each of these forms of aggression (Delville et al. 2000; Choi et al. 2005). Although frequently characterized as a negative social interaction, aggression plays a highly constructive role in the formation of social relationships. In the vast majority of mammals that do not form pair bonds, dominance relationships provide social bonds that serve many adaptive functions (e.g., resource distribution) and that ultimately reduce social conflict. Typically, dominance relationships are rapidly determined through aggressive interactions but are primarily maintained through social communication (e.g., scent marking, vocalization), thereby reducing the dangers associated with intense fighting (Albers et al. 2002; Fernald 2014).

4.3.1 Oxytocin and Competitive Behavior

Some of the earliest evidence that Oxt is involved in both aggression and social communication came from studies in squirrel monkeys. In male squirrel monkeys, with established dominant–subordinate relationships, i.c.v. administration of Oxt increases aggression in dominant males, while having no effect in subordinate males (Winslow and Insel 1991b). In contrast, Oxt stimulates scent marking in subordinate males but does not alter scent marking in dominant males (Winslow and Insel 1991b), thus demonstrating that social experience can determine the behavioral response to Oxt acting in the brain. Studies in rats suggest one mechanism that might contribute to the effects of social experience on the behavioral response to Oxt. Dominant male rats have significantly higher levels of Oxtr mRNA in the MeA 3 h after the social encounter that defined their relationship. Further, infusion of an Oxtr antagonist immediately after the establishment of subordinate status increases the duration of the dominant–subordinate relationship (Timmer et al. 2011). Social status also significantly impacts the circulating levels of serum Oxt in primates . In rhesus monkeys, dominant females have significantly higher levels of serum Oxt than subordinates (Michopoulos et al. 2011, 2012). The importance of social experience in determining the behavioral response to Oxt/Avp is a theme seen repeatedly in this chapter.

Historically, there have been little data supporting a role for Oxt in the regulation of intermale aggression in laboratory species of rodents . However, some recent work suggests that pharmacological treatment with Oxt may have antiaggressive effects in adult males. Work from Jaap Koolhaas’ laboratory has found that pharmacological enhancement of Oxt in rats, by intranasal treatment or i.c.v. infusion, reduces offensive aggression and promotes prosocial exploratory behaviors (Calcagnoli et al. 2013, 2014, 2015a). Furthermore, these inhibitory effects seem to be mediated by Oxt acting via the Oxtr in the CeA; however, it should be noted that blocking endogenous Oxt signaling in the CeA has no effect (Calcagnoli et al. 2015b). Thus, it is proposed that these findings should be considered in the context of pharmacological effects rather than being directly regulated by the local endogenous Oxt system (Calcagnoli et al. 2015b).

The role of Oxt in the neural regulation of female offensive aggression is sparse; as mentioned previously, females of the most commonly studied laboratory rodents, unlike many other mammalian species, rarely display aggressive behaviors outside of the peripartum period. However, there is evidence in female Syrian hamsters that Oxt can influence competitive behaviors. Specifically, microinjection of Oxt into the MPOA-AH reduces offensive aggression, and injection of an Oxtr antagonist increases offensive aggression directed toward a female intruder (Harmon et al. 2002a). Traditionally, studies of Oxt in female rodents have focused on maternal aggression. This is a unique, and transient, hormonal/physiological time in a female’s life and is characterized by high levels of nurturing behaviors directed toward pups and aggressive behaviors directed toward intruders. During the peripartum period, Oxt has anxiolytic effects (Bosch and Neumann 2008), specifically via its actions in the PVN and CeA (Blume et al. 2008; Jurek et al. 2012; Windle et al. 1997; Huber et al. 2005; Viviani et al. 2011; Knobloch et al. 2012). These decreases in anxiety permit females to attend to their pups. But even with lower levels of anxiety and strong bonds with her offspring, dams can display high levels of maternal aggression. Depending on the brain area and the behavioral state of the animal, Oxt and/or Oxtr antagonists can either increase or decrease maternal aggression (for review, please see Bosch 2013). For instance, in rats bred for low-anxiety rats, Oxt in the PVN increases maternal aggression (Bosch et al. 2005), but when microinjected into the BNST of Wistar rats decreases maternal aggression (Consiglio et al. 2005). In hamsters, injection of Oxt into the amygdala facilitates maternal aggression (Ferris et al. 1992a). So, while central Oxt is important to aggression in females, its effects are context and site specific. There are also numerous studies that support the assertion that developmental exposure to Oxt is important for the proper development of motivated social behaviors, including aggression , but those studies will not be reviewed in this chapter (recently reviewed in Miller and Caldwell 2015; Hammock 2015).

4.3.2 Vasopressin and Competitive Behavior

Most of what we know about the role of nonapeptides in competitive behavior comes from studies of Avp and aggressive behavior. Further, much of the work has focused on the Avpr1a, as this was the first centrally identified receptor, and as such, there are a number of pharmacological tools available. However, the Avpr1b appears also to be important for displays of aggressive behavior. Because of the differences in the distribution of these two receptor subtypes, this section will be divided by subtype.

4.3.3 The Vasopressin 1a Receptor and Competitive Behavior

The first evidence for a role for Avp in aggression came from studies in male hamsters, in which the injection of Avpr1a antagonists into the AH significantly inhibited offensive aggression (Ferris and Potegal 1988; Potegal and Ferris 1990). Subsequent studies have replicated these results and shown that Avp injected into the AH stimulates offensive aggression (Caldwell and Albers 2004a; Ferris et al. 1997). However, the ability of Avp to stimulate aggression by its action in the AH appears to depend on an individual’s prior social experience. Avp injected into the AH increases aggression in male hamsters previously trained to fight other hamsters and in hamsters socially isolated for at least four weeks, but not in hamsters housed in social groups. The ability of social experience to enhance the response of the AH to Avp appears to be mediated by experience-dependent increases in the number of Avpr1a in the AH (Albers et al. 2006; Cooper et al. 2005). In summary, Avp has powerful effects on offensive aggression in male hamsters, but only if social experience has upregulated Avpr1a receptors in the AH.

Avp can also have potent effects on aggression in male prairie voles through a similar mechanism. Sexually naïve male voles are essentially non-aggressive, choosing to explore intruder males as opposed to attacking them (Winslow et al. 1993). Following mating-induced pair bonding , males display high levels of aggression toward conspecifics other than their mate and have increases in the density of Avpr1a receptors in the AH (Gobrogge et al. 2009). Further, overexpression of the Avpr1a in the AH using viral vector gene transfer increases aggression in non-pair-bonded males. Thus, in male hamsters and prairie voles, species with very different types of social organization, an individual’s social experience can modulate the number of Avpr1a in the AH and thereby regulate the intensity of aggression that is expressed.

Another hypothalamic region where Avp influences aggression is the ventrolateral hypothalamus (VLH). Avp injected into the VLH facilitates aggression in gonadally intact males and castrated males given testosterone but does not facilitate aggression in castrated controls (Delville et al. 1996). Avpr binding is also reduced in the VLH following castration and precastration levels of binding can be restored by testosterone. In contrast, it is not known whether testosterone influences the ability of Avp to alter aggression when injected into the AH. However, since castration reduces Avpr binding within this region (Delville et al. 1996), it seems possible that castration could reduce aggression stimulated by Avp. It should be noted that the relationship between gonadal hormones and aggression is complex; however, it is clear that testosterone does not simply induce aggression in males (for review, see Johnson et al. 1995; Young et al. 2000; Demas et al. 2007). In females, while there is evidence that gonadal hormones can affect Avpr1a binding within the VLH (Delville and Ferris 1995), the specific effects on aggression are unknown.

There are also extra-hypothalamic regions where aggression and Avp activity have been linked. In both male rats and mice selected for varying levels of aggression, a negative correlation has been observed between Avp fiber density in the LS and the amount of intermale aggression (Compaan et al. 1993; Everts et al. 1997). Interestingly, these differences in Avp and aggression are not related to differences in circulating levels of testosterone (Elkabir et al. 1990). The role of septal Avp in aggression has also been studied in male rats bred for low or high anxiety. Release of Avp into the LS is significantly lower in the much more aggressive low-anxiety rats than in the high-anxiety rats that exhibit lower levels of aggression. In addition, septal administration of Avp to the highly aggressive group and administration of the Avpr1a antagonist to the low aggressive group did not alter the levels of aggression expressed (Beiderbeck et al. 2007). The level of aggressiveness and the pattern of Avp expression in the LS and BNST are also correlated in mice. The monogamous California mice (Peromyscus californicus) have shorter attack latencies and increased Avp-ir in the BNST and LS compared to the polygamous, white-footed mice (Peromyscus leuopus) (Bester-Meredith et al. 1999). Interestingly, when California mice pups are cross-fostered to white-footed mice dams, they are less aggressive in adulthood than those reared by the same species, and they have less Avp-ir in the BNST and SON compared to controls (Bester-Meredith and Marler 2001). These data in mice, like those from hamsters, suggest that changes in the social environment are able to alter Avp neurocircuitry and the behavior driven by that circuitry. In addition, although relationships between aggressiveness and Avp expression and release within the LS have been found, Avp may not have any direct effects on male aggression by its actions in the LS.

Despite the powerful effects of Avp on aggression in the hypothalamus, not all central manipulations of Avp have been found to influence offensive aggression. Although i.c.v. injections of an Avpr1a antagonist increase the latency to the onset of aggression in highly aggressive California mice, i.c.v. injections of Avp or Avpr1a antagonists have no effect on aggression in white-footed mice (Bester-Meredith and Marler 2001). In non-pair-bonded male prairie voles with extensive experience with aggression, i.c.v. administration of an Avpr1a antagonist does not inhibit aggression (Winslow et al. 1993). There is other evidence from rats that i.c.v. administration of Avp does not alter the expression of aggression nor does deletion of the Avpr1a receptor in mice (Elkabir et al. 1990; Wersinger et al. 2007b). While it is clear that Avp can stimulate Aggression by its action in some brain sites, it remains possible that Avp might act to reduce aggression by its action in other brain regions or by its action on other receptors (e.g., the Oxtr) to reduce/inhibit aggression. It is also important to recognize that aggression is a complex behavior and may be facilitated by neurochemical systems other than Avp, at least in some cases.

In females, Avp has very different effects on offensive aggression than it does in males. As described above, in male hamsters, injection of Avpr1a antagonists into the AH inhibits offensive aggression and injection of Avp stimulates aggression (Ferris and Potegal 1988; Potegal and Ferris 1990; Caldwell and Albers 2004a; Ferris et al. 1997). In contrast, in female hamsters, an Avpr1a antagonist injected into the AH stimulates offensive aggression and injection of Avp inhibits aggression in the resident–intruder test (Gutzler et al. 2010). More recently, similar sex differences have been seen in the effects of Avp and Avpr1a antagonists on social play behavior in rats (Bredewold et al. 2014; Veenema et al. 2013). For example, injection of Avpr1a antagonists in the LS increases social play in juvenile males and reduces social play in juvenile females. In a related study, a negative correlation was found between Avp mRNA levels in the BNST and social play in male juvenile rats (Paul et al. 2014). As social play is hypothesized to be a precursor to aggressive behavior, these data are consistent with Avp-related sex differences in the regulation of competitive behaviors.

4.3.4 The Vasopressin 1A Receptor and Maternal Aggression

Maternal aggression is an intense form of aggression displayed by lactating mothers confronted by intruders (Lonstein and Gammie 2002). Studies using Avpr1a −/− mice found that maternal aggression does not differ from that seen in wild-type mice (Wersinger et al. 2007b). In contrast, i.c.v. administration of Avp in lactating rats reduces and an Avpr1a antagonist increases aggression toward male intruders (Nephew and Bridges 2008; Nephew et al. 2010). The effects of i.c.v. administration of Avp and an Avpr1a antagonist on maternal aggression were also examined in rat strains that had been selectively bred for high (HAB) or low (LAB) anxiety (Bosch and Neumann 2010). In both strains, Avp was found to increase and an Avpr1a antagonist to decrease maternal aggression, respectively. Other studies have used microdialysis to examine the role of Avp in specific brain regions in the regulation of maternal aggression in HAB and LAB rats (Bosch and Neumann 2010). In HAB, but not LAB, rats, Avp is positively correlated with maternal aggression in the CeA but not the PVN. In addition, administration of an Avpr1a antagonist into the CeA reduces aggression in HAB rats, while administration of Avp into the CeA increases aggression in LAB rats. The ability of Avp to promote maternal aggression by acting in the CeA does not appear to be restricted to HAB rat strains since similar results have been reported in Sprague-Dawley rats (Meddle and Bosch, unpublished; cited in, Bosch 2011). In the future, it will be important to clarify the effects of Avp on maternal aggression , its site(s) of action, and whether the effects of Avp on aggression are related to anxiety levels.

4.3.5 The Vasopressin 1b Receptor and Competitive Behavior

There is compelling evidence that the Avpr1b is essential for displays of aggressive behavior directed toward a conspecific (for review, see Caldwell et al. 2008a, c; Stevenson and Caldwell 2012). In resident–intruder and neutral cage aggression tests, Avpr1b −/− mice display fewer attacks and have longer attack latencies than Avpr1b +/+ controls (Wersinger et al. 2002, 2007a). Further, in a reversal of a resident–intruder test where the experimental mice are intruders rather than residents, Avpr1b −/− mice display normal defensive avoidance behaviors when attacked by a stimulus animal, but are less likely to initiate retaliatory attacks (Wersinger et al. 2007a). Pharmacological studies using the Avpr1b antagonist, SSR149415, support the findings of work in Avpr1b −/− mice. Syrian hamsters orally administered SSR149415 have reductions in the frequency and duration of offensive attacks, in chase behaviors, in flank marking, and in the olfactory investigation that often precedes and accompanies an offensive attack (Blanchard et al. 2005). Mice given SSR149415 display fewer defensive bites when forced to encounter a threatening predator and reductions in the duration of offensive aggression in a resident–intruder test (Griebel et al. 2002).

The deficits in aggressive behaviors observed in Avpr1b −/− mice are not limited to males. Following parturition, female Avpr1b −/− mice have reductions in maternal aggressive behaviors, compared to control mice, as measured by longer attack latencies and fewer attacks, directed toward a male intruder (Wersinger et al. 2007a). Interestingly, the disruption of the Avpr1b does not affect all forms of aggressive behavior. In a nonsocial context, such as the predation of a cricket, Avpr1b −/− and Avpr1b +/+ mice have similar attack latencies (Wersinger et al. 2007a). Based on the genetic and pharmacological data, it has been hypothesized that the disruption of the Avpr1b does not specifically disrupt aggressive behavior, but rather the ability to have the appropriate behavioral response within a given social context (Caldwell et al. 2008c; Young et al. 2006; Stevenson and Caldwell 2012).

With prominent expression within the pyramidal neurons of the CA2 region of the hippocampus, recent work has focused on what the Avpr1b may be doing here. To this end, Pagani et al. 2015 have shown that replacement of the Avpr1b in the dorsal CA2 region of Avpr1b −/− mice restores socially mediated attack behaviors. Further, selective Avpr1b antagonists result in the production of N-methyl-D-aspartic-acid-dependent excitatory postsynaptic responses specifically within the CA2 region of control mice, but not Avpr1b −/− mice (Pagani et al. 2015). While the hippocampus is not currently a recognized node in the SBNN , it is a part of the motivational pathway. The CA2 region is structurally unique, as it does not receive rich mossy fiber input from the dentate gyrus (Tamamaki et al. 1988), and is the only part of the hippocampus to receive input from the posterior hypothalamus and the perforant pathway (Bartesaghi et al. 2006; Borhegyi and Leranth 1997; Vertes and McKenna 2000), which connects the entorhinal cortex to the hippocampal formation (Bartesaghi and Gessi 2004). Further, there is a vasopressinergic projection from the PVN to the CA2 region (Cui et al. 2013). Based on the findings described here, we, and others, have hypothesized that the CA2 region of the hippocampus may aid in the formation and/or recall of accessory olfactory-based memories (Caldwell et al. 2008c; Young et al. 2006). Thus, it seems likely that this is a region that will need to be included in future discussions of the SBNN and its interactions with the mesolimbic DA system.

4.4 Social Communication

While Avp plays a key role in the regulation of social communication in hamsters (see below), Oxt also contributes to its regulation. Hamsters communicate using a form of scent marking called flank marking, and the expression of flank marking is essential for the maintenance of dominant/subordinate relationships (Johnston 1985). After the rapid establishment of dominance, aggressive behavior declines and flank marking increases in dominant hamsters and to a lesser extent in subordinate hamsters (Ferris et al. 1987). In the absence of flank marking, aggression remains high and a stable relationship is not formed.

Oxt injected into the areas extending from the MPOA to the posterior medial and lateral aspects of the AH (referred to from here on as the MPOA-AH) of dominant female hamsters induces flank marking in a dose-dependent manner but only when the dominant hamsters are tested with their subordinate partners (Harmon et al. 2002b). Oxt does not induce flank marking when injected into the MPOA-AH of socially naive female hamsters tested with an opponent or alone. In males, by contrast, Oxt induces flank marking in dominant hamsters when they are tested with their subordinate partner or alone (Harmon et al. 2002b). These data indicate that social experience, social context, and sex interact to regulate the ability of Oxt to stimulate flank marking by its actions in the MPOA-AH in hamsters.

Although Oxt can stimulate flank marking, Avp plays the predominate role in regulating its expression. Avp stimulates high levels of flank marking in male and female hamsters by acting on the Avpr1a in the rostral hypothalamus (Ferris et al. 1984, 1985; Albers et al. 1986). It is of note that the MPOA-AH is substantially larger than the site where Avp can induce aggression (Ferris et al. 1986a). Avp can also induce flank marking following its injection into the LS, BNST, and PAG (Irvin et al. 1990; Hennessey et al. 1992). Gonadal hormones modulate the ability of Avp to stimulate flank marking by regulating the number of hypothalamic Avpr1a (Huhman and Albers 1993; Albers et al. 1988). In the LS, BNST, and PAG, gonadal hormones have only small effects on the ability of Avp to stimulate flank marking, suggesting that the MPOA-AH may be the primary site where gonadal hormones influence the ability of Avp to stimulate flank marking (Albers and Cooper 1995). Interestingly, Avp seems to induce flank marking regardless of the social context. For example, in hamsters with an established dominant/subordinate relationship, injection of Avp produces high levels of flank marking in either the dominant or subordinate hamster during social interactions (Ferris et al. 1986b). Similar results have been obtained in squirrel monkeys where Avp injected i.c.v. induces scent marking in both dominant and subordinate males (Winslow and Insel 1991a).

4.5 Interactions Between Oxytocin, Vasopressin, and Dopamine in the Regulation of Cooperation/Competition

As discussed in detail above, Oxt and Avp act within brain regions considered to be components of the mesolimbic DA system to influence cooperative and competitive behaviors. There is also evidence that DA in the mesolimbic DA system plays important roles in the regulation of both cooperative and competitive behaviors. Non-selective DA antagonists block mating-induced partner preferences in both male and female prairie voles, and treatment with the non-selective DA agonist apomorphine facilitates partner preference in the absence of mating (Aragona et al. 2003; Wang et al. 1999). The NAcc shell, and not the core, appears to be the site of action for these drugs since local administration into the NAcc shell, but not the core, has the same effects as systemic administration. Increases in DA activity within the NAcc that occurs following mating is necessary for the formation of pair bonds in male prairie voles (Aragona et al. 2003). Activation of DA D2 receptors in the NAcc can induce a pair bond in cohabiting voles in the absence of mating, while activation of D1 receptors can block pair bonding (Aragona et al. 2006). In addition, D1 receptors are increased in the NAcc in male prairie voles following the formation of a pair bond. Mate guarding aggression in males can also be inhibited by D1 antagonists injected into the NAcc (Aragona and Wang 2009). Other regions in the network also play a key role in the formation of cooperative behavior. Pair-bonded voles have lower concentrations of D1 receptors and higher concentrations of D2 receptors in the medial prefrontal cortex than non-pair-bonded voles (Smeltzer et al. 2006). Enhancement of DA release in the VTA via administration of GABA or glutamate antagonists can also induce pair bonding in male voles (Curtis and Wang 2005).

There is also considerable evidence that the mesolimbic DA system plays a critical role in the regulation of competitive behavior and in particular aggression (for review, see de Almeida et al. 2005). For instance, the non-selective DA receptor agonist apomorphine stimulates aggression and flank marking in male hamsters (Hyer et al. 2012). Social experience can regulate the expression of DA in several nodes of the SBBN. The amount of the rate-limiting synthetic enzyme for DA, tyrosine hydroxylase, increases significantly in several regions of the SBNN including the LS and BNST as well as within the shell of the NAcc in males trained to fight as compared to controls (Schwartzer et al. 2013). In addition, activation of D1 and D2 receptors in the NAcc influences the expression of aggression and its rewarding properties (Couppis and Kennedy 2008; Miczek et al. 2002); although selective aggression in male voles appears to require only the D1 receptor activation (Aragona et al. 2006). While DA is involved in various aspects of the preparation, expression, and consequences of aggression, its precise role remains elusive.

The importance of interactions between Oxt, Avp, and DA in social motivation has been widely discussed, and yet there are few studies demonstrating a direct link between these neuropeptides and DA in controlling social behavior. Partner preferences induced by the activation of D2 receptors in the NAcc can be prevented by administration of an Oxtr antagonist and partner preference induced by i.c.v. Oxt administration can be blocked by a D2 antagonist administered in the NAcc (Liu and Wang 2003). In addition, overexpression of Avpr1a in the VP of the non-pair-bonded meadow voles results in mating-induced partner preferences that can be blocked by a D2 antagonist given prior to mating (Lim et al. 2004).

Despite the mounting empirical evidence suggesting that Oxt and Avp can directly interact with DA to influence social behavior, the majority of evidence for this interaction comes from studies showing either that Oxt and/or Avp act within structures that comprise the mesolimbic DA system, or that manipulations of DA can influence the same social behaviors that are influenced by Oxt and Avp. Interestingly, there are very little data on whether Oxt or Avp might contribute the rewarding properties of social behavior. A limited amount of data suggest that Oxt can induce a conditioned place preference (CPP) when given peripherally to male rats or centrally to female mice (Liberzon et al. 1997; Kent et al. 2013). Recently, we investigated whether injection of Oxt or Avp into the VTA could produce a conditioned place preference in male hamsters (Song et al. 2014a). Both Oxt and Avp increase CPP when injected into the VTA. The administration of selective Oxt and Avpr1a agonists and antagonists revealed that the rewarding properties of both Oxt and Avp in the VTA are mediated by the Oxtr and not the Avpr1a.

5 Cooperativity and Competitiveness in Humans

As we all know, cooperation and competition are a hallmark of nearly all human endeavors. Successful cooperation and competition require a set of social skills collectively termed social cognition, which allow an individual to engage in social behaviors that are appropriate for a particular context. Since the emergence of the social cognition field in the late 1960s and early 1970s, there has been a concerted scientific effort to understand the complicated cognitive processes that underlie human social interactions. While social cognition and many disciplines that are brought to bear in this field are interesting, there is accumulating evidence that Oxt and Avp in humans are important to social cognition. This section will highlight what we know about Oxt and Avp in humans, in particular how exogenous administration of these nonapeptides impacts measures of social cognition.

As described throughout this chapter, there is considerable compelling evidence that Oxt promotes social behaviors, at least in specific contexts. In humans, the study of social behaviors includes testing procedures designed to measure trust, the ability to read facial expressions, the memory for socially salient information, such as faces, and more recently measures of empathy. In most human studies focused on the therapeutic effects of Oxt and Avp, they are exogenously administered intranasally. This delivery system is preferable as it is considered noninvasive and some assert that Oxt and Avp are able to cross the blood–brain barrier (Born et al. 2002); however, this latter assertion is questionable given their size, hydrophilic nature, as well as numerous other issues.

5.1 Nonapeptides and Social Cognition in Healthy Humans

5.1.1 Oxytocin and Social Cognition in Humans

Investigation into the role of Oxt in human cognition has dramatically increased over the last decade. More recently, attempts have been made to reconcile the existing data on the role of Oxt into a broader theory and understanding of human cognition. For example, De Dreu proposes that Oxt plays a critical role in the motivation of cooperation and competition in humans (De Dreu and Kret 2015; De Dreu 2012). This hypothesis stems from the idea that humans are social animals and are likely to cooperate with others, even with those genetically unrelated. He suggests that Oxt motivates humans to like and empathize with others in their group, to comply with group norms, and to reciprocate trust with other group members, while competing with out-group members. While De Dreu’s hypothesis is in reference to the endogenous Oxt system, this idea is supported by some recent work using intranasal Oxt. Essentially, intranasal Oxt increases “in-group” favoritism when individuals are asked to use intuitive decision making; however, Oxt has an opposite effect if individuals are asked to use reflective decision making (Ma et al. 2015). These data suggest that an individual’s “cognitive style” may contribute to the effects of Oxt, which has implications for both endogenous and exogenous Oxt.

Given the literature in animal models, and the proposed prosocial effects of Oxt, in recent years, there has been a surge in the number of clinical studies that have administered exogenous Oxt to improve social interactions in healthy individuals. While the data do suggest that intranasal Oxt as a therapeutic agent has some efficacy, long-term and dose–response studies have yet to be completed. In humans, intranasal Oxt may influence the processing of social information in several ways, including selective attention, enhancement of the memory, and/or the appraisal of socially relevant information (Guastella and MacLeod 2012). Intranasal Oxt also increases trust, as measured by an individual’s willingness to accept social risk during a social interaction (Zak et al. 2005; Kosfeld et al. 2005). However, similar to what is observed in other species, the effects of Oxt on trust are nuanced and often sex specific. For instance, if subjects are provided with information that suggests that a trustee is untrustworthy, then intranasal Oxt does not facilitate trust, or as the authors state, “Oxt makes people trusting, not gullible” (Mikolajczak et al. 2010). In females that have had their trust betrayed, intranasal Oxt results in less restoration of trust behavior compared to controls (Yao et al. 2014). However, in males, intranasal Oxt does not alter trust behavior in individuals that have had their trust betrayed, whereas placebo controls decrease their trust in response to betrayal (Baumgartner et al. 2008). Further, when intranasal Oxt treatment in males is coupled with functional magnetic resonance imaging (fMRI), there is a reduction in activity in areas of the brain associated with processing fearful stimuli, such as the amygdala and some areas of the midbrain, and reward feedback, such as the striatum (Baumgartner et al. 2008).

Oxt also has sex-specific effects in the context of reciprocal altruism, as measured using the prisoner’s dilemma game, which examines cooperative exchange (Chen et al. 2015b; Feng et al. 2014; Rilling et al. 2014). Specifically, intranasal Oxt administered to females causes them to treat computer partners more like human partners; this same effect is not observed in males (Rilling et al. 2014). In this same context, there are also striking sex differences in neural activation, as measured by fMRI. In males, Oxt increases activation in the caudate/putamen while decreasing activation in females. The authors suggest that in this context, Oxt may increase the reward or salience of positive social interactions among males, while having the opposite effect in females (Feng et al. 2014; Rilling et al. 2014). Recent work also indicates that Oxt selectively improves kinship recognition in women but not men and that Oxt improves men’s performance in competition recognition (Fischer-Shofty et al. 2013). Taken together, these data suggest that intranasal Oxt does not universally promote social interactions, but rather that the effects are subtle and dependent upon the social context and sex of the individuals. As mentioned previously, there has been a flood of clinical studies using intranasal Oxt, and with those studies, there is evidence that it can improve the ability to infer another individual’s mental state, improves facial recognition memory, alters the processing of faces, and can facilitate empathy (Heinrichs et al. 2009; Rimmele et al. 2009; Domes et al. 2007; Guastella et al. 2008, 2009; Zak et al. 2007; Hurlemann et al. 2010; Unkelbach et al. 2008; Bos et al. 2015).

While the intranasal administration of Oxt can definitely influence aspects of social cognition , it is the individual differences in endogenous Oxt and their association with social cognition that is equally, if not more, important. Not only are individual differences in Oxt genetically heritable (Rubin et al. 2014), but also they appear to be stable (Feldman et al. 2007) and can be modulated by social interactions (Feldman et al. 2010). Further, individual differences in Oxt and Oxt signaling may increase an individual’s vulnerability to certain neuropsychiatric disorders characterized by altered social cognition; see the section titled “Implications for Neuropsychiatric Disorders.” Some recent work suggests that endogenous Oxt may underlie natural variations in social perception (Lancaster et al. 2015). Further, in a more recent study, plasma Oxt is associated with increases in activity in brain areas that are important to social cognition such as the superior temporal sulcus, inferior frontal gyrus, and medial prefrontal cortex (Lancaster et al. 2015). Interpretation of some of the aforementioned data is muddled by our lack of understanding about the functional relationship between peripheral and central Oxt. But, it does follow that peripheral physiology and central physiology (behavioral output) are often going to be coupled, even if the systems are separate. However, while there is some evidence that they can be associated with one another (Landgraf and Neumann 2004), this does not always appear to be the case (Amico et al. 1990; Rosenblum et al. 2002; Winslow et al. 2003; Leng and Ludwig 2015).

5.1.2 Vasopressin and Social Cognition in Humans

The role of Avp in the regulation of social behavior in humans has not been studied as extensively as Oxt, though it is often associated with antisocial rather than prosocial behaviors. In males, intranasal Avp increases electromyogram (EMG) activity to socially neutral facial expressions (Thompson et al. 2004), as well as the memory for happy and angry faces (Guastella et al. 2010b). This suggests that Avp acts to bias an individual to perceive a neutral stimulus as an aggressive or threatening stimulus or enhances the encoding of negative social cues (Thompson et al. 2004; Guastella et al. 2010b). In females, Avp decreases EMG responses to happy and angry faces, suggesting that Avp acts to increase the perception of friendliness (Thompson et al. 2006). These sex-specific responses to Avp are supported by other studies as well (Feng et al. 2014; Rilling et al. 2014; Thompson et al. 2006) and may reflect sex differences in Avp neurochemistry and in the types of behavioral strategies employed during social interactions. Other effects include decreases in emotional recognition in men (Uzefovsky et al. 2012) and increases in empathy concern in individuals who received more “parental warmth” in their early family life (Tabak et al. 2015). It should be noted that the same issues that plague intranasal Oxt studies, such as the separation of central and peripheral Avp, are issues with the aforementioned studies as well.

5.2 Oxytocin, Vasopressin, and the Mesolimbic Dopamine System

Studies employing nasally administered Oxt and Avp have found numerous examples where these peptides alter the activity in various structures of the mesolimbic DA system. For example, Oxt augments the ventral striatum response to viewing the faces of romantic partners and to reciprocated cooperation from human partners (Rilling et al. 2012; Scheele et al. 2013). In women, Oxt increases activity in the VTA in response to both positive and negative facial expressions, but reduces the VTA response to reciprocated cooperation (Groppe et al. 2013; Chen et al. 2015a). While it is clear that Oxt and Avp can significantly alter activity in several structures in the mesolimbic DA system, the functional significance of these responses is not known. It is also possible that these neuropeptides alter social motivation without engaging DA neurons. For example, recent data suggests that Oxt enhances the attractiveness of female faces and correlates with increased activity in the striatum, while having no measureable impact on DA signaling (Striepens et al. 2014). However, more sensitive methods may be needed to detect subtle changes in behaviorally evoked DA release.

6 Implications for Neuropsychiatric Disorders

Oxt and Avp have also been implicated in a variety of neuropsychiatric disorders, particularly those characterized by aberrant social interactions and/or heightened aggression , such as ASD, personality disorder, schizophrenia, and posttraumatic stress disorder (PTSD). Further, many of these neuropsychiatric disorders are also characterized by dysfunctions of the mesolimbic DA system (for review, see Dichter et al. 2012). As many of these disorders are complex and have multiple etiologies, it becomes even more important not only to understand the potential contributions of Oxt and Avp, as well as their therapeutic effects, but also to determine how these systems interact with the DA system.

6.1 Autism Spectrum Disorder

ASD is characterized by repetitive behaviors, communication difficulties, and abnormal sociability (Matson and Nebel-Schwalm 2007a, b). In preclinical models of ASD, specifically Oxt −/− and Oxtr −/− mice, behavioral deficits are observed that are consistent with some of the symptoms of ASD (Crawley et al. 2007; Lee et al. 2008a, b; Macbeth et al. 2009; Wersinger et al. 2008; Winslow and Insel 2002; Ferguson et al. 2000, 2001). Evidence that Oxt may have a role in ASD comes from several sources. There are reports of reduced concentrations of Oxt in the cerebral spinal fluid (CSF) of autistic children, and reduced CSF Oxt is correlated with impairments in social functioning (Modahl et al. 1998). There are also increases in the amount of the Oxt prohormone in the blood of autistic children, which is indicative of incomplete processing of Oxt into its biologically active form (Green et al. 2001). Oxt treatment in adults with ASD results in the reduction of repetitive behaviors and improvements in emotional recognition (Hollander et al. 2003, 2007) and can increase eye gazing, a behavior important to social communication (Auyeung et al. 2015). In youth and adults with ASD, intranasal Oxt enhances emotion recognition (Domes et al. 2013, 2014; Guastella et al. 2010a) and increases social interactions (Andari et al. 2010). Further, in adults with Asperger’s syndrome, Oxt-mediated increases in emotional recognition are associated with an increase in activity in the left amygdala (as measured in fMRI) (Domes et al. 2014).

There are also some genetic and epigenetic links between the Oxt system and ASD. There are data in the Chinese Han population, in Finnish families, in Caucasian children, and in individuals with “high-functioning” ASD, suggesting that portions of the OXTR gene may contain susceptibility loci for ASD (Wu et al. 2005; Ylisaukko-oja et al. 2006; Jacob et al. 2007; Wermter et al. 2010; Nyffeler et al. 2014). Epigenetic modifications of the OXTR gene have also been reported, with hypermethylation of the OXTR promoter found in ASD subjects and subsequent reductions in OXTR mRNA (Gregory et al. 2009). A more recent study focused on whether the levels of DNA methylation of the OXTR could predict individual variability in social perception found that there are significant associations between the degree of OXTR methylation and social perception (Jack et al. 2012). Though the sample sizes in these latter two epigenetic studies are small, the data are provocative and will likely facilitate more research in this area.

Data implicating Avp in the etiology of ASD are sparse, but there have been studies suggesting that polymorphisms of the AVPR1A may contribute to ASD (Kim et al. 2002; Wassink et al. 2004; Yirmiya et al. 2006; Kantojarvi et al. 2015; Tansey et al. 2011). Further, two of the polymorphisms, RS3 and RS1, have been linked to differential activation of the amygdala (Meyer-Lindenberg et al. 2009), providing a possible neural substrate with which the Avp system may interact to mediate a genetic risk for ASD.

6.2 Personality Disorder

Personality disorder is characterized by disconnect between an individual’s behavior and cultural norms. Those diagnosed with personality disorder have impairments in at least two of the following areas: (1) cognition, (2) affectivity, (3) interpersonal functioning, and (4) impulse control (American Psychiatric Association 2013). To date, only a handful of studies have examined how exogenous Oxt treatment affects individuals diagnosed with personality disorder, and they have been performed primarily in individuals diagnosed with borderline personality disorder (BPD). Females diagnosed with BPD and treated with intranasal Oxt have reductions in stress reactivity (Simeon et al. 2011), decreases in hypersensitivity to threat, and decreases in amygdala activation in response to angry faces compared to controls; suggesting that Oxt in these individuals may decrease their reaction to a perceived social threat (Bertsch et al. 2013; Brune et al. 2013). This same research group also reports that females with BPD have reduced plasma Oxt concentrations (Bertsch et al. 2012). There are also data that suggest that intranasal Oxt can worsen trust in BPD and that this worsening is correlated with the patients’ history of childhood trauma (Ebert et al. 2013). One issue with all of the aforementioned studies is that BPD patients were not used as controls, only healthy individuals, making it more difficult to parse out what is going on specifically within BPD patients. However, these data do reinforce the importance of past history and social context.

One study has measured Oxt in the CSF of individuals with personality disorder, which includes individuals diagnosed with intermittent explosive disorder, as well as BPD. This study found that while CSF Oxt concentrations are not correlated with having a personality disorder, a life history of suicidal behavior is inversely correlated with Oxt (Lee et al. 2009b). The authors suggest that these data are consistent with the previous work in animal models demonstrating that Oxt can reduce aggressive behaviors (Consiglio and Lucion 1996; Giovenardi et al. 1998; Harmon et al. 2002a; Bales and Carter 2003).

Since individuals diagnosed with a personality disorder often have more impulsive behaviors, which can result in increased aggression , it is not surprising that Avp has been examined in these individuals. However, the data appear to be contradictory. A study by Coccaro et al. (1998) found a positive correlation between Avp in the CSF of personality-disordered individuals that have a life history of aggressive behavior. However, an earlier study found no differences in CSF Avp between violent offenders and controls (Virkkunen et al. 1994). It may be that differences in the populations studied account for the inconsistency in the findings, but it seems that more work in this area is warranted.

6.3 Schizophrenia

There are three broad categories of symptoms that characterize schizophrenia: (1) positive (e.g., hallucinations and delusion), (2) negative (e.g., anhedonia, impaired social behavior), and (3) cognitive/attentional (e.g., impaired memory and executive function) (American Psychiatric Association 2013). However, in humans, while its role has remained controversial, Oxt has been linked to schizophrenia since the 1970s when it was used as an antipsychotic (Bujanow 1974, 1972; for a recent review, see Rich and Caldwell 2015). Altered CSF concentrations of Oxt are reported in patients diagnosed with schizophrenia (Beckmann et al. 1985; Linkowski et al. 1984). However, the data are conflicting with some studies reporting an increase in Oxt and the Oxt carrier protein neurophysin I (Linkowski et al. 1984; Beckmann et al. 1985) and another reporting no change in CSF Oxt concentrations (Glovinsky et al. 1994). However, patients with higher plasma levels of Oxt have less severe positive symptoms and exhibit fewer social deficits (Rubin et al. 2010, 2011).

There are also reports of SNPs in the promotor regions of the OXT and OXTR genes that may contribute to symptom severity and treatment efficacy in schizophrenic patients (Teltsh et al. 2012; Watanabe et al. 2012; Montag et al. 2013). SNPs of the OXTR gene are associated with the severity of symptoms and the improvement of the positive symptoms of schizophrenia following treatment with antipsychotics (Souza et al. 2010a, b). Additionally, postmortem analysis of brain tissue from unmedicated schizophrenia patients has altered immunoreactivity of the Oxt carrier protein, neurophysin I, in the PVN, internal palladium, and substantia nigra (Mai et al. 1993). Most recently, in patients with schizophrenia and polydipsia, decreases in plasma Oxt were found to correlate with the ability to correctly identify facial emotions (Goldman et al. 2008). Thus, it appears that alterations in the Oxt system underlie all three symptom domains. Given the dysregulation of the Oxt system in patients with schizophrenia, Oxt has been studied as a candidate for use as a therapeutic.

Some studies suggest that Oxt may have antipsychotic properties (for review, see Macdonald and Feifel 2012; Bakermans-Kranenburg and van Jzendoorn 2013). Previous work found that injections of Oxt can reduce the symptoms of psychosis and anhedonia in patients with schizophrenia (McEwen 2004; Churchland and Winkielman 2012). In healthy patients, intranasal Oxt increases holistic processing, divergent thinking, and creative cognition (De Dreu et al. 2013), and studies in schizophrenic patients report that intranasal Oxt can be beneficial. Specifically, intranasal Oxt can facilitate social cognition (Davis et al. 2013; Feifel et al. 2010; Pedersen et al. 2011; Averbeck et al. 2012) and alleviate some of the cognitive deficits and positive symptoms (Pedersen et al. 2011). Yet, intranasal Oxt may be most effective as an adjunctive therapy to already prescribed antipsychotics, where chronic Oxt treatment is able to ameliorate some of the negative symptoms and the cognitive deficits, as well as the positive symptoms (Feifel et al. 2010, 2012; Modabbernia et al. 2013). While this research suggests that Oxt treatment has the potential to improve symptoms in all three domains, where in the brain and how these effects are mediated remains unknown.

Support for a potential role for Avp in schizophrenia comes from studies indicating that treatment with neuroleptics improves psychiatric symptoms and reduces (or normalizes) Avp in blood plasma (Peskind et al. 1987; Raskind et al. 1987). In studies using an animal model that lacks Avp, the Brattleboro rat, there are reports of deficits in behaviors associated with schizophrenia, specifically social discrimination and prepulse inhibition of the startle reflex; these deficits can be rescued following treatment with antipsychotics (Feifel and Priebe 2001, 2007; Feifel et al. 2004, 2007, 2009).

Given that schizophrenia is a complicated multiple etiology neuropsychiatric disorder, it may be that Oxt and Avp only contribute to certain types of schizophrenia. Further, it is likely that their action within specific parts of the brain is particularly important. Thus, the use of preclinical models continues to be critical to help improve our understanding of the specific neural substrates where these neuropeptides may act to contribute to the symptoms associated with schizophrenia as well as to determine their therapeutic efficacy (for review, see Rich and Caldwell 2015; Feifel 2011, 2012; Rosenfeld et al. 2010).

6.4 Posttraumatic Stress Disorder

Work examining Oxt and Avp in the context of PTSD has only recently begun, but PTSD is characterized by some symptoms related to Oxt and Avp function. Specifically dysregulation of stress reactivity and problems with intimate relationship interactions, which relies on normal social cognition (American Psychiatric Association 2013; Monson et al. 2009). While there have been no clinical studies looking at Oxt, intranasal Oxt has been proposed to be used as an adjunctive treatment to help enhance psychotherapy, as it may help ultimately reduce the fear response at the level of the amygdala and diminish the hormonal stress response (Koch et al. 2014). In regard to Avp, there has been one clinical study. Intranasal Avp administered to men, but not women, diagnosed with PTSD results in improved social cognition with their heterosexual partner. Further, men’s urinary Avp is negatively correlated with the severity of their PTSD (Marshall 2013). While these data are limited in scope, they do suggest that understanding the role of Avp in modulating social cognition may have clinical relevance in some psychiatric conditions.

7 Conclusions and Future Directions

Oxt and Avp are key neurochemical signals that act throughout the brain to influence socially motivated behaviors, having powerful effects on both cooperative and competitive behaviors mediated by their actions within the SBNN . There is also a substantial body of evidence that the mesolimbic DA system is involved in regulating cooperative and competitive behaviors. More recently, these two networks have been combined into a larger social decision-making network. However, there is a lack of understanding about the neurochemical linkages between the “social” and “motivational” elements within these networks. We propose that the Oxt and Avp systems represent a bridge between the social and motivational elements. More specifically, we propose that Oxt and Avp provide the critical links between specific elements of the SBNN and DA within the motivational network that produce the forces that drive social behaviors. While there is little direct support for this hypothesis, there is ample evidence that Oxt and Avp can act within structures comprising the mesolimbic DA system and that manipulations of DA can influence the same social behaviors as Oxt and Avp.

These interactions also have important implications for neuropsychiatric disorders characterized by aberrant social behaviors, particularly those with known disruptions in Oxt and Avp signaling. Rigorous testing of preclinical models continues to be the best way to uncover the specific mechanisms (i.e., neurochemistry, substrates, and circuits) that are important to socially motivated behaviors. Thus, it is crucial that a variety of species be studied to help determine both the conserved and unique mechanisms of Oxt and Avp actions across development, sexes, and behaviors. Overall, this is an exciting time in the field of socially motivated behaviors, with the advent of new experimental tools and an improvement in our ability to examine entire circuits we are now on the brink of making significant leaps in our understanding of these systems.