Keywords

1 Introduction

Most social information is not processed by specialized sensory systems but through the main sensory channels that are also used to perceive inanimate objects. Efficient extraction of sensory information from conspecifics is critical to social recognition across perceptual boundaries throughout evolution (Brennan and Kendrick 2006). Social recognition has been classically studied in rodents and other mammals (Fleming et al. 1979; Sanchez-Andrade and Kendrick 2009; Wiesner and Sheard 1933). Many species rely heavily on the emission and detection of olfactory cues for social recognition. Social recognition appears to be modulated via different mechanisms than the recognition of non-social objects (Ferguson et al. 2000). One modulator that appears to set brain circuits in particular states for processing of social information is the neuropeptide oxytocin (OT) (Insel and Young 2001; Kendrick et al. 1997).

In this chapter, we will focus on the processing of social olfactory signals and their modulation through OT. We will look at the olfactory processing from the perspective of brain circuits based on the currently known mechanisms and identify open questions on sensory processing of social information. We will highlight the particularities in the processing of social cues and the current knowledge on larger olfactory circuits employed by different types of social interactions.

1.1 Anatomy of the Main Olfactory System

We will first describe the anatomical organization of the main olfactory system. Odorants are inhaled during the breathing cycle into the nasal cavity where they get in contact with the olfactory epithelium. The olfactory epithelium harbors the olfactory sensory neurons. Olfactory sensory neurons project to neuropil structures in the MOB called glomeruli, where they make direct or indirect contact with a number of interneurons and the MOB projection neurons. MOB projection neurons are comprised of mitral cells (MCs) and middle tufted cells that directly convey sensory information to the olfactory cortices (Fig. 1). Olfactory cortex neurons are thus just two to three synapses away from the peripheral sensory neurons. MCs have an apical dendrite that targets one glomerulus and several lateral dendrites that extend up to a few millimeters horizontally. Middle tufted cells also have an apical dendrite that targets preferentially a single glomerulus and lateral dendrites, which are shorter than those of MCs.

Fig. 1
figure 1figure 1

Oxytocin in the main olfactory system. Top: Odorants are detected in the main sensory epithelium (MOE) that project to the main olfactory bulb (MOB) where they innervate mitral cells (MC) that then directly project to the anterior olfactory nucleus (AON), ventral striatum (VS), piriform cortex (PCx), amygdala (AMY), and lateral entorhinal cortex (EC). Bottom: The table shows the density of oxytocin terminals and receptors in the respective region (++, high; +, moderate; +, low;?, unknown). Also relevant oxytocin effects on sensory processing on behavior have been identified in the respective brain region (+) or are unknown (?)

Full size image

The activity of MC and middle tufted cells (hereafter collectively referred to as MCs) is modulated by two major groups of interneurons: periglomerular interneurons and granule cell (GC) interneurons. In the mouse, there are approximately 40,000 MCs compared to 50,000 periglomerular interneurons and three million GCs. The MOB is organized in different layers, determined by the position of neuronal cell bodies or neuropil. The two MOB layers with the largest volume are the GC layer close to the center of the MOB, where the cell bodies of GCs are located, and the external plexiform layer, a neuropil layer located between the MC layer and the glomeruli. The external plexiform layer contains mainly synapses between GCs and the lateral dendrites of MCs. The apical dendrites of MCs receive input from olfactory sensory neurons. The information is back-propagated along the lateral dendrites and integrates inhibition mediated through reciprocal synaptic contacts with GCs. These reciprocal dendro-dendritic synapses contain a glutamatergic excitatory output synapse emerging from MC dendrites and a GABAergic inhibitory synapse back from the GC onto the same MC dendrite. Activity invading the lateral dendrites of MCs elicits lateral inhibition of other MCs via dendro-dendritic connections through GCs. Lateral inhibition of MC is thought to be involved in odor discrimination and synchronization of rhythmic MCs activity (Margrie et al. 2001). MCs send their axons to large number of higher brain areas. These higher brain areas comprise the primary olfactory cortices, namely the anterior olfactory nucleus (AON) and anterior and posterior piriform cortex as well as the amygdala and entorhinal cortex and the olfactory tubercle of the ventral striatum.

GCs are the most abundant cells in the MOB and are axon-less inhibitory interneurons that have a basal and an apical dendrite. The branched parts of the apical dendrite are covered with spines containing bidirectional dendro-dendritic synapses described above. The basal dendrite and the unbranched initial parts of the apical dendrite receive glutamatergic input from axon collaterals of MCs and the olfactory cortex (Balu et al. 2007). The MOB receives strong top-down projections from central brain regions. The first group of top-down projections is glutamatergic and originates from the AON and to a lesser extent from other olfactory cortices. Most parts of these cortical top-down projections provide input to the GCs of the MOB (Balu et al. 2007; Brunjes et al. 2005; Cajal 1911). These top-down inputs are transiently active in a brain-state dependent manner and increase GC firing and thereby can modulate inhibition on MC cells (Balu et al. 2007; Boyd et al. 2012; Markopoulos et al. 2012; Oettl et al. 2016). The second group of top-down projections releases neuromodulators, such as noradrenaline, serotonin, or acetylcholine (Devore and Linster 2012).

1.2 Anatomy of the Accessory Olfactory System

The accessory olfactory system (AOS), through its close connections with endocrine centers of the hypothalamus, is usually conceived to be involved in the detection of odorants that influence puberty, estrous induction, or pregnancy block (Keverne 2004). The sensory neurons of the accessory olfactory system are located in the vomero-nasal organ (VNO) (Fig. 2). The VNO is a blind-ended tube at the base of the nasal septum. To sample non-volatile molecules, many mammalian species engage in direct physical contact with scent sources of conspecifics (Luo et al. 2003). Compared to the MOB, surprisingly little is known about the anatomy and coding of the attached accessory olfactory bulb (AOB); with existing studies highlighting fundamental differences in the neuronal organization (Larriva-Sahd 2008) and coding of AOB neurons (Luo et al. 2003). AOB neurons output primarily to the medial nucleus of the amygdala (von Campenhausen and Mori 2000) and several hypothalamic and limbic regions involved in the regulation of reproductive behavior (Scalia and Winans 1975). AOB outputs thus bypass higher cognitive cortical centers.

Fig. 2
figure 2figure 2

Oxytocin in the accessory olfactory system. Top: Odorants are also detected in the vomero-nasal organ (VNO) that projects to the accessory olfactory bulb (AOB) where axons terminate on AOB projection neurons that directly output to various hypothalamic nuclei (HYP) and parts of the amygdala (AMY). Bottom: The table shows the density of oxytocin terminals and receptors in the respective region (++, high; +, moderate; +, low;?, unknown). Also relevant oxytocin effects on sensory processing on behavior have been identified in the respective brain region (+) or are unknown (?)

Full size image

1.3 Organization of Oxytocin Receptors and Projections in the Olfactory System

Oxytocin is provided to the olfactory brain regions through axonal projections from the paraventricular nucleus of the hypothalamus (Knobloch et al. 2012). This study found no evidence for OT fibers in the MOB, but at high densities in the AON. Relatively high axon densities are also found in the medial and central amygdala, while the olfactory tubercle and entorhinal cortex had relatively low fiber densities. Oxytocin receptors (OTRs) are highly expressed in the olfactory system of rodents. We will focus here primarily on mRNA expression in adult male rats for which the most detailed analysis exists (Vaccari et al. 1998); comparable results have, however, also been made for receptor autoradiography (Numan and Insel 2003) and immunohistochemistry in mice (Mitre et al. 2016). In rat MOB, OTR mRNA is expressed at comparably low levels in the GC layer, MC layer, and also in some periglomerular cells (Vaccari et al. 1998). OTR binding is relatively highest in the GC layer with faint OTR binding in the glomerular layer (Ferguson et al. 2000; Ferris et al. 2015). OTR transcripts are also present in the olfactory tubercle of the ventral striatum. The AON and piriform cortex expressed high levels of OTR mRNA. Interestingly, OTR immunoreactivity in the posterior piriform cortex was found to be higher in female than male mice (Mitre et al. 2016). It appears important to note here that species-specific distribution of brain OTR seems to determine the behavioral patterns of a given species (Insel and Shapiro 1992) and inter-individual expression differences in brain regions may explain variance in behavior (Calcagnoli et al. 2014; Olazabal and Young 2006). Finally, OTR expression is regulated by hormonal states with OTR mRNA expression increases in the MOB and medial amygdala transiently with parturition (Meddle et al. 2007).

Two recent studies also examined the functional recruitment of olfactory brain regions to OT. Functional MRI studies allow the measurement of brain activation patterns through detection of changes in the blood flow (BOLD) that correlates to neuronal activity (Logothetis and Wandell 2004). In female post-partum rats, robust activation was observed in the MOB and heavy clustering of positive BOLD voxels was observed in brain areas that receive direct synaptic inputs from the MOB (Febo et al. 2005). Specifically, intracerebroventricular OT administration and suckling of pups that elicits OT release activate the AON, piriform, entorhinal and prefrontal cortex, olfactory tubercle, and amygdala. A more recent rodent MRI study also examined differences between “peripheral” (i.e., intraperitoneal) and “central” OT administration in adult male rats (Ferris et al. 2015): “peripheral” OT affected all subdivisions of the MOB in addition to the cerebellum and several brainstem areas relevant to the autonomic nervous system. “Peripheral” OT administration results in similar BOLD changes in the MOB as well as the AON and cortical amygdala. Hence, “peripheral” OT can efficiently act on MOB activity and downstream circuits through yet unknown mechanisms.

2 Olfaction-Dependent Forms of Social Interaction and Olfactory Memories

2.1 Olfaction for the Recognition of Individuals and Triggering of Sexual Behaviors

All social behavior involves some form of social (re)cognition. Social cognition involves the sensing, incorporation, integration, and recognition of information about conspecifics and allows an animal to react appropriately to social stimuli across a variety of contexts (Insel and Young 2001). Social recognition refers to the processes through which an animal recognizes another animal as familiar. Failure to respond appropriately to what should be a familiar social stimulus may involve a disruption of social recognition with improper integration of multiple contextual cues in higher level processing. It appears therefore important to separate social recognition from subsequent responses when considering complex social behavior.

Social recognition involves an initial sensing of the subject through one or several modalities, memory formation, and the eventual remembrance of the subject in subsequent encounters. Through social recognition, an appropriate response may enhance or decrease olfactory investigation, aggression or affiliation, fighting or escape, depending on their previous experience with the now recognized individual (Wacker and Ludwig 2012). For instance, mother sheep recognize their own offspring and allow only them to suckle milk (Kendrick et al. 1992). Rats can differentiate colony members from strange intruders, with aggression occurring selectively against male intruders but not amongst colony members (Blanchard et al. 1988).

Recognition memory may be considered to be composed of two components: long-lasting memory of salient information about a recognized stimulus and shorter lasting familiarity of a recently encountered stimulus. The short- and long-lasting recognition memories are not as distinct as they may initially sound. Short- and long-term recognition memory may rather build on each other. Recognition of a lamb by its mother is associated with long-term recognition that forms very quickly after birth and can initially be disrupted by separating the pair (Kendrick et al. 1997). Here, recognition memory only survives longer separation periods when the lamb and its mother had already spent longer periods of time together. Other forms of recognition also display short- and long-term retention of individual recognition, as detailed later for olfaction-dependent recognition among members in rat colonies.

Social olfactory recognition involving the MOS has been studied mainly in the context of offspring recognition in sheep (Kendrick et al. 1997) and social recognition in rodents (Ferguson et al. 2000; Thor and Holloway 1982). A special case of social recognition is that of mate recognition in pair-bonding voles (Insel and Young 2001). Here the act of mating in conjunction with odorant stimuli from the partner leads to a recognition memory for the partner as well as promoting the formation of a pair bond. This has also been shown to involve the MOS and has been reviewed in detail in Chap. 5 of this volume by Bosch and Young.

2.2 Same- and Other-Sex Social Interaction

Tests of social recognition in rodents rely on the intrinsic motivation of animals to investigate other individuals in a social context and particularly novel ones. Rodents investigate novel conspecifics more than familiar ones (Thor and Holloway 1982). During the encounter, a mouse or rat will intensely investigate a novel conspecific by sniffing the head and ano-genital region. On a second encounter, after a given time interval, it will investigate the animal significantly less. However, if a new animal is presented, the investigation duration goes back up to initial levels. The reduced investigation following repeated encounters is taken as a measure of social recognition memory. This paradigm was classically used as a model for short-term olfactory memory, since it lasts no more than an hour in singly housed animals (Bluthe and Dantzer 1990). However, a more recent study showed that housing conditions and isolation can affect the duration of social recognition memories (Kogan et al. 2000). Group-housed, but not singly housed, mice can retain social recognition memory for up to 7 days after a single encounter with another individual.

Central OT modulates social memory formation of male and female rats (Benelli et al. 1995; Engelmann et al. 1998). Low doses of intracerebroventricular OT prolong social memory from 1 to 2 h (Benelli et al. 1995; Engelmann et al. 1998). However, OT applied during the recognition phase had no influence (Benelli et al. 1995; Engelmann et al. 1998; Lukas et al. 2013). Also, OT and OTR knockout mice show deficits in the recognition of conspecifics; again, in OT knockout mice, intracerebroventricular administration of OT reversed the deficit before, but not after, exposure to the first conspecific (Ferguson et al. 2000; Choleris et al. 2003; Takayanagi et al. 2005). In rats, the prolonged recognition of juvenile males through OT administration appears to be effective through applications of OT in the MOB (Dluzen et al. 1998). Conversely, OTR blockers applied to the MOB (Larrazolo-Lopez et al. 2008), medial amygdala (Choleris et al. 2007), or the lateral septum (Lukas et al. 2013) impair same-sex recognition. OT-deficient male mice display normal initial other-sex exploration, but impaired recognition that could be rescued by OT infusion into the amygdala (Ferguson et al. 2001). It may need to be considered here that some brain circuits in mutant mice may have adapted more than others during development to this lack of OT signaling and that OT modulates recognition at different levels of processing. In further support of OT modulating olfactory sampling and recognition, optogenetically evoked endogenous OT release enhanced the initial ano-genital exploration of conspecifics in adult rats and prolonged recognition memory for that conspecific in subsequent encounters (Oettl et al. 2016). Also, direct administration of OT into the MOB lengthens retention time in a social discrimination test in adult male rats (Dluzen et al. 1998). It is, however, possible that differential effects are obtained depending on the dose/volume and site of injected OT within the MOB. Interestingly, the AON, with its high levels of OTR expression, is nearby and can be reached through diffusion by OTR agonists or blockers. As detailed further below, the AON turns out to be a potent site for OT-dependent modulation of social recognition and odor processing in the MOB through recruitment of cortical top-down projections (Oettl et al. 2016). Also, interactions of vasopressin and OT and their potential receptor cross-talk (Lukas and Neumann 2013) need to be investigated in the MOS. Taken together, the existing data reveal the MOB as an essential target to understand OT in social interactions.

2.2.1 Oxytocin Actions on Top-Down Projections from the Anterior Olfactory Cortex

Sensory processing in the MOB is heavily influenced by cortical top-down projections from the AON (Markopoulos et al. 2012; Kay and Laurent 1999). Compared to the MOB, the AON receives dense OT fibers and has high levels of OTR expression (Knobloch et al. 2012; Vaccari et al. 1998). Also, the AON displays pronounced immediate early gene expression to all kinds of social encounters that require formation of recognition memory (Kim et al. 2015). Indeed, initial exploration of conspecifics and subsequent recognition memory are impaired if OTRs are deleted in the AON of adult mice (Oettl et al. 2016). In the AON network, endogenous OT release and OTR agonists transiently increase the intrinsic excitability of AON regular-firing neurons and the excitatory synaptic drive. GCs are the main target of AON axonal projections to the MOB. Indeed, AON excitation through OT propagates through top-down projections to increase glutamatergic synaptic input to GCs. In vivo putative GCs display also transient increases in their firing rate following OTR activation in the AON. Along with GCs innervating MCs, OT in the AON increases the inhibitory drive to MCs. Thus, these observations provide a pathway for OT to increase the excitability of AON top-down projections that drive MOB interneurons for inhibition of MCs. Compatible with OT promoting information extraction, OTR activation in the AON in vivo enhances the signal-to-noise ratio by lowering baseline firing of MCs and by increasing their peak odor responses (Oettl et al. 2016). Thus, OT in the AON, that efficiently recruits GCs in the MOB, has similar effects as direct GCs excitation (Alonso et al. 2012). Actions of OT on neural coding were recently also observed in hippocampal in vitro recordings (Owen et al. 2013) with OT enhancing the signal-to-noise ratio in hippocampal spike transmission by modulating interneurons. It is therefore possible that modification of information transfer through induction of high signal-to-noise states is a shared feature of OT in different systems. OT release in the auditory cortex, amygdala, and hippocampus also work primarily through modulation of interneuron activity (Owen et al. 2013; Huber et al. 2005; Marlin et al. 2015).

The GC-MC network has features that allow for increasing firing of stimulus-driven inputs while suppressing weak activity as occurs during baseline firing, depending on the strength of inhibition from GCs. MCs are neurons with burst firing properties. Increases in inhibitory input within a certain range bring voltage-dependent conductances in a different state so that the regenerative conductances result in more intense burst discharges in response to a given stimulus (Angelo et al. 2012; Balu and Strowbridge 2007). OT effects become globally weaker with more moderate OTR recruitment in the AON, but still increase the signal-to-noise of odor responses (Oettl et al. 2016) with inhibition of MC baseline firing becoming negligible. As predicted from biophysical properties of MCs, that subtly increase GC inhibition boost stimulus-driven odor responses of MCs (Angelo et al. 2012; Balu and Strowbridge 2007), moderate OTR activation in the AON still increase peak firing responses to odorants. Compatible with the larger dynamic range of initially small odor responses, moderate recruitment of top-down projections through OT preferentially amplified those weaker odor responses. Stronger GC recruitment continues to increase burst odor responses and then also significantly reduces baseline firing of MCs (Alonso et al. 2012). Indeed, stronger OTR activation in the AON reduced background firing and amplified peak odor responses of MCs. Finally, in line with increases in GC inhibition only boosting stimulus-driven burst responses, MCs that did not respond to an odorant before OTR activation in the AON also did not respond to that odorant following recruitment of top-down projections.

The OT-induced state in MOS processing is predicted to promote stimulus selection and information extraction and thereby may facilitate memory formation. Barlow (1961) predicted two aspects that relate to the here observed OT actions, i.e., sparsening in MC cell coding of sensory information and the existence of sensory relays that modulate the flow of information according to requirements of other parts of the brain. This second concept matches the top-down control of early sensory information flow with respect to the current state of the animal during brain-wide modulation through OT. The MOS is used both for social and non-social information processing. OT is released preferentially during interactions with conspecifics (Lukas and Neumann 2013) and is therefore predicted to primarily affect social cues. Indeed, OTR deletion in the AON selectively affected olfaction-dependent sampling behavior and recognition of conspecifics, but not odor discrimination or recognition outside a social context. Together, these observations argue for a modulatory system that is specialized to come into action for sensory processing of social information and may modify salience of social cues (Shamay-Tsoory and Abu-Akel 2016). These findings support the premise that one of the major functions of central OT is to bring multiple levels of sensory, motor, and emotion regulating systems into a state for social interaction. Exploration of conspecifics is promoted through more intense olfactory sampling of conspecifics (Oettl et al. 2016) and a low anxiety state induced by non-sensory systems (Lukas et al. 2011; Viviani et al. 2011). At the level of sensory processing, OT modifies the state of early olfactory presentation that may enhance salience of concurrently presented odorants and help to detect relevant information of conspecifics during social encounters. Compatible with this idea, a possible reason for the longer conspecific exploration times in mice with OTR deleted in the AON (Oettl et al. 2016) could be less efficient information extraction due to OT’s effects on the gain of odor representations. Through its cortical top-down projections into the early olfactory system, OT modifies the global gain control of olfactory coding before MOB output spreads into divergent higher-order pathways including the posterior piriform cortices, the ventral striatum (olfactory tubercle), the amygdala, and the entorhinal cortices. Many of these higher-order brain regions are activated during social interactions and also express OTRs (Vaccari et al. 1998; Kim et al. 2015; Dolen et al. 2013) allowing for further modifications of information through OT during particular types of social behavior. Before discussing findings relevant to higher brain regions, we will now first examine the role of the two parallel olfactory systems.

2.2.2 The Main and Accessory Olfactory System Pathways in Social Recognition and Sexual Behaviors

The segregation of molecule types processed by the AOS and MOS is not black-and-white in that the AOS primarily processes non-volatile social odor cues, while the MOS deals with volatile, non-social odorants (Scalia and Winans 1975). Urine, like most other social olfactory stimuli, consists of a large number of distinct chemical compounds that vary according to the sex, strain, social, and physiological status of the emitter (Jemiolo et al. 1989). Some volatile odorants trigger sexual response behaviors along with traditional “pheromone” concepts, but are detected by the MOS (Dorries et al. 1995) and persist if the AOS is lesioned (Cohen-Tannoudji et al. 1989). In turn, the vomero-nasal neurons are sensitive to volatile urinary molecules (Del Punta et al. 2002; Leinders-Zufall et al. 2000). Some non-volatile social cue molecules like major histocompatibility complex peptides are processed in parallel by the AOS and MOS (Spehr et al. 2006) with even more sensitive activation of sensory neurons in main olfactory epithelium (Spehr et al. 2006; Leinders-Zufall et al. 2004). In support of this parallel pathway, functional MRI revealed activity changes in the AOB and the MOB to volatile urine odorants (Xu et al. 2005). However, lesioning the MOS completely suppressed the preference for both volatile and non-volatile opposite-sex olfactory cues observed in both Y-maze and habituation/dishabituation tasks in either sex (Baum and Keverne 2002; Keller et al. 2006; Ma et al. 2002; Wesson et al. 2006). Conversely in the same tasks, lesions of the AOS were not effective in disrupting mate recognition in mice of either sex, again using volatile and non-volatile body and urine scents (Keller et al. 2006; Jakupovic et al. 2008; Pankevich et al. 2004, 2006). Thus, MOS and AOS can detect in part overlapping sets of non-volatile and volatile social cues even though both systems are not necessarily required for eliciting behaviors to the respective cue.

Whole-brain immediate early gene expression following same- and other sex social interactions can provide additional insights into the involved circuits. For instance, in male mice, brain regions downstream of the vomero-nasal epithelium revealed a strong bias toward the female interaction-evoked brain activation, including the AOB, part of the cortical amygdala and the entire medial amygdala (Kim et al. 2015). In contrast, male–male interaction induced activation in fewer AOB-linked areas. Male–female and male–male interaction-evoked brain activation revealed largely overlapping immediate early gene induction among MOB-connected brain regions, including the dorsal MOB, the AON, piriform cortex, cortical amygdala, and lateral entorhinal cortex (Kim et al. 2015; Wacker et al. 2010). Tenia tecta and the postpiriform transition area were selectively activated by male–female interactions and stronger recruitment was observed in ventral striatum (olfactory tubercle and nucleus accumbens shell) and orbital medial prefrontal cortices compared to male same–sex interactions. Thus, male same-sex stimuli showed activation of all MOB-linked structures but only a subset of the AOB-linked structures.

Functional distinction of the AOS and MOS has also been inferred from the direct innervation of the medial amygdala, primarily through AOB outputs (Scalia and Winans 1975); see, however, (Kang et al. 2009; Luskin and Price 1983). Medial amygdala may be rather considered a place where signals from the MOS and AOS can converge (Wacker and Ludwig 2012; Brennan and Zufall 2006; Canteras et al. 1995). A recent study examined synaptic plasticity in AOB-amygdala connections in a social recognition paradigm (Gur et al. 2014). A protein synthesis inhibitor applied to the medial amygdala did not affect short-term, but a form of long-term social recognition test. To examine potential correlates of this form of long-term memory, synaptic plasticity was examined by electrical stimulation to either the AOB or the MOB in anesthetized rats. While AOB stimulation evoked strong, prolonged direct responses in the medial amygdala, MOB only evoked weak and short responses consistent with poor direct innervation (Luskin and Price 1983). The induced long-term depression was strongly augmented on intracerebroventricular administration of OT before plasticity induction; the enhanced depression was blocked by an OTR antagonist. When OT was applied without the plasticity-inducing electrical burst stimulation in the AOB, the amplitudes of AOB input to medial amygdala increased. How far these forms of plasticity contribute to long-term social recognition needs to be elaborated.

2.3 Parental Recognition of Their Offspring

The role of the olfactory system in offspring recognition has been most extensively studied in sheep and rats. We will therefore focus on these two species, since differences in mothering style can be observed. Mothers of some species, like rodents, build nests in which they give birth to a large number of offspring that have immature sensory and motor systems. The immature newborns barely leave the nest, limiting the mothers need to recognize individual pups. In contrast, other species, like sheep, tend to have a small litter of fully developed young capable of following the mother shortly after birth. Consequently, mothers of these species develop discriminative maternal care favoring their own young, allowing them to suck while rejecting any alien young that may approach. In this respect, the establishment of a selective bond within the first few hours after parturition represents one of the essential characteristics of maternal behavior in sheep (Levy and Keller 2009).

The initial typical response of the majority of virgin female rats to pups is avoidance. As in several other species, the majority of virgin rats are repelled by placenta and/or amniotic fluid (AF) that cover the neonate (Kristal and Graber 1976; Levy et al. 1983). These inhibitory processes that depress maternal responsiveness in virgin rats are affected by endocrine changes occurring at parturition, with recently parturient rats readily accepting pups (Slotnick et al. 1973). Following lesioning the MOB and/or AOB, these aversive properties are eliminated and, consequently, females exhibit a rapid onset of maternal behavior (Fleming et al. 1979; Fleming and Rosenblatt 1974a, b). Also, anosmic female sheep with lesioned MOS are neither repelled nor clearly attracted to AF (Levy et al. 1983), while lesions of the VNO are without effect (Levy et al. 1995b). Lesions of the medial amygdala or ventromedial hypothalamus disinhibit maternal retrieving in virgins (Fleming et al. 1979, 1980 Bridges et al. 1999; Numan et al. 1993). Thus, maternal behavior emerges when avoidance of the pups decreases and motivation to approach them increases (Rosenblatt and Mayer 1995). Olfactory inputs play an essential function in these two motivational systems. In sheep, the main factor controlling the olfactory shift from repulsion to attraction toward AF is the process of delivery itself through inducing of OT release (Da Costa et al. 1996; Kendrick and Keverne 1992; Lévy et al. 1992). Pedersen and Prange (1979) were the first to describe that synthetic OT induces spontaneous maternal care in steroid-primed virgin female rats when infused into the lateral ventricle. In confirmation of a role of endogenous OT, intracerebroventricular infusion of antiserum raised against OT (Pedersen et al. 1985) or of an OTR blocker (Fahrbach et al. 1985) impairs the onset of maternal care in steroid-primed virgin female rats. In addition, anosmic virgin rats frequently become maternal with intracerebroventricular OT administration, supporting that access to key sensory stimuli interacts with the brain OT system to facilitate the onset of maternal care (Wamboldt and Insel 1987).

The second aspect involves the recognition and bond formation to the offspring. The ewe forms a selective olfactory memory in a sensitive period in the first 4 h after parturition and will consequently reject the approach of a strange lamb (Kendrick et al. 1992). Both the sensitive period for odor learning and the maternal acceptance behavior are dependent on the hormonal state during late gestation and triggered by vagino-cervical stimulation during parturition. A post-partum ewe can even be induced to accept, and form recognition memory for, a strange lamb by brief vagino-cervical stimulation up to 3 days after giving birth (Kendrick et al. 1991). Interestingly, experienced ewes are more efficient in establishing individual recognition in successive births (Kendrick 1994; Keverne et al. 1993), possibly due to modifications in the MOB and OT systems (Broad et al. 1999).

In pregnant sheep MOBs, the majority of MCs respond preferentially to food odorants, but not to lamb or AF odorants (Kendrick et al. 1992). However, after birth, a substantial number of MCs respond preferentially to lamb odorants, supporting the idea of a change in salience of the lamb odorant. These shifts in electrical responsiveness of MCs are paralleled by concurrent changes in the release of GABA and glutamate (Kendrick et al. 1992), noradrenaline and OT in the MOB and attached circuits (Levy et al. 1995a). OT infused into the MOB of ovariectomized, estrogen-primed virgins induces a rapid onset of maternal behavior (Yu et al. 1996). Virgin rats treated with a regimen of hormones designed to mimic the parturitional changes in progesterone and estradiol exhibit a preference for pup-related odorants (Fleming et al. 1989). Laboratory strains of rats and mice do not form a selective bond with their own young and they retrieve also alien young. Nevertheless, when given the choice, mother rats retrieve their own young first before taking care of alien pups, a preference abolished by MOB lesioning (Rosenblatt and Lehrman 1963).

In summary, while there appears to be no functional specificity of either the MOS or AOS in the onset of maternal behavior among species, only the MOS is implicated when individual odor discrimination of the young is required. Neural structures, such as the MOB, undergo profound changes when exposed to offspring odorants at parturition. These changes in synaptic circuitry contribute both to maternal responsiveness to these odorants and to their memorization (Levy and Keller 2009). The data also indicate that, in rats as in sheep, experienced mothers are able to use multiple channels of information and to compensate for the loss of one type of sensory cues.

2.4 Olfaction in Association with Learning in a Social Context

Social olfactory learning is not limited to recognition of odor cues signaling individuality; it can also involve odorants not emitted by the interaction partner itself. A recent study (Choe et al. 2015) also provided evidence that OT plays a crucial role in the formation of learned associations between odorant and socially significant cues through pairing of an olfactory conditioned stimulus with a social unconditioned stimulus. OT was required for social learning but was dispensable for learning tasks that do not involve social cues. The social-specific effects of OT are consistent with other rodent studies (Ferguson et al. 2000; Oettl et al. 2016; Dolen et al. 2013). Piriform cortex is enriched for OTRs in female mice and OT wash-in increased spontaneous rates of excitatory and inhibitory synaptic events in the piriform cortex (Mitre et al. 2016). In contrast to the auditory cortex and hypothalamus, OT greatly increased the rate of spontaneous inhibition onto excitatory piriform neurons and blocking excitation prevented oxytocinergic disinhibition in the piriform cortex, suggesting that the mechanism of OT modulation can partially differ across brain areas. Such disinhibition can be effective for inducing long-term synaptic modifications in the auditory cortex. OT promotes association learning with aversive as well as appetitive social cues to sensory stimuli and recruitment of piriform ensembles (Choe et al. 2015). These results suggest that OT conveys saliency of social stimuli to sensory representations in the piriform cortex during odor-driven social learning. Thus, OT appears to mediate social learning of opposing valence, depending on the context, compatible with previous observations that OT is released by both mating and aggressive encounters in rodents (Waldherr and Neumann 2007). In summary, OT is poised to influence social learning at multiple loci from perception of both conditioned and unconditioned stimuli to behavioral output. How OT coherently orchestrates these distributed circuits to produce social learning remains to be determined.

2.5 Oxytocin on System Actions Relevant to Olfaction

OT also appears to modulate olfaction-related processes like the intensity of ano-genital conspecific exploration (Oettl et al. 2016) or sniffing (Wesson 2013). Sniffing influences the acquisition of odorants through modulation of respiratory behavior (Uchida et al. 2006) during motivated and social behaviors (Doty 1986). Sniffing may, however, also transmit social communication since rats investigating the facial region of another conspecific often elicit a decrease in sniff rate in the conspecific (Wesson 2013) and depends on the rat’s social status. Here, reciprocal decreases in sniffing frequency in subordinate rats could reflect a submissive behavior elicited in response to dominant rat investigation. Intraperitoneal OT treatment in rats with established social hierarchies abolished agonistic behaviors and reciprocal sniffing displays. As previously described, OT significantly reduced the aggression scores of dominant rats (Calcagnoli et al. 2014). Conversely, OT in previously submissive rats resulted in reduced suppression in sniffing of the submissive rat. Interestingly, rats rendered unable to smell still display reciprocal sniffing behavior. Together, these findings demonstrate that rodents utilize sniffing behaviors communicatively, not only to collect but also to convey social information.

3 Summary and Perspective

To many species, olfaction is the dominant sense for social recognition and triggers sexual and parenting behaviors. No wonder that OT in these species is preferentially clustered in the olfactory system. The last decades have substantially revised the view of the respective contributions of the MOS and AOS pathways. In many species, the MOS plays an almost exclusive role for recognition of individuals and, perhaps more surprising, the MOS also triggers some sexual behaviors through volatile and non-volatile cues. Considering the widespread role of OT in lower and higher brain regions, the requirement of OT in modulation of early sensory processing is evident. Yet, a number of questions need to be elaborated or further developed. We need a better understanding in the brain regions that store and retrieve social recognition memories and how OT modifies the underlying circuit functions. This may help to dismantle the interactions of OT acting in multiple brain regions. In this context, we also need more detailed information as to when OT is released during social interactions. Finally, the mechanisms need to be further elaborated regarding how OT sets sensory systems in a particular state for processing of social cues. These mechanistic insights may then explain recent findings on olfaction as a potential trait marker of autism spectrum disorders (Rozenkrantz et al. 2015). This study assumed that there are brain templates for sensory-motor coordination underlying diverse behaviors that are impaired in autism spectrum disorder (Haswell et al. 2009). Olfaction relies on sniffing that is an internal action model, where sniff magnitude is automatically modulated by the valence of odorants (Arzi et al. 2014). Autistic children had a profoundly altered sniff response, sniffing equally regardless of odor valence and allowed for a high diagnostic value that was correlated to severity in social, but not motor impairment. The first results support that olfaction is altered in human psychiatric disorders already in young children. Importantly, the sniff response is similar across humans and rodents (Mandairon et al. 2009) and may provide an entry point for translational research.