Introduction

The involvement of serotonin (5-HT) in anxiety and affective disorders has been highlighted by the successful use of serotonin-modulating drugs in the treatment of clinical conditions such as generalized anxiety, panic disorder, obsessive compulsive disorder and depression (Maes and Meltzer 1995; Graeff et al. 1996; Graeff 2002). However, despite intensive research, there are many controversies about the exact role 5-HT plays in these psychopathologies.

In an attempt to interpret the existing evidence about the involvement of 5-HT in anxiety and depression, Deakin and Graeff (1991) hypothesized that different 5-HT pathways and receptor subtypes modulate the neural substrates of depression, panic and generalized anxiety. At the core of this proposal are the two major 5-HT-containing cell groups of the midbrain, the dorsal (DRN) and median (MRN) raphe nuclei.

According to the cited authors, the ascending 5-HT pathway that originates in the DRN and innervates the amygdala and frontal cortex would facilitate defensive behaviors that are expressed in response to potential or distal threats, e.g., inhibitory avoidance. On the other hand, the DRN pathway that innervates the dorsal periaqueductal gray matter (DPAG) would inhibit flight or fight reactions in response to proximal danger. Dysfunction of these pathways would lead to generalized anxiety and panic disorder, respectively. Finally, the pathway connecting the MRN to the hippocampus would increase resistance or tolerance to chronic unavoidable stress and failure in this coping mechanism would result in depression.

A wealth of experimental and clinical evidence reviewed elsewhere (Deakin et al. 1994; Graeff 1993, 2002) gives support to this hypothesis. However, some aspects of this theory deserve further consideration. By sending massive projections to the hippocampus (Vertes et al. 1999), the MRN has also been implicated in the functioning of Gray’s behavioral inhibition system (for a review, see Gray and McNaughton 2000) and, hence, in anxiety processing. Accordingly, it has been demonstrated that whereas lesions of this nucleus generate signs of behavioral disinhibition indicative of anxiety reduction (Jacobs et al. 1974; Srebo and Lorens 1975; Jacobs and Cohen 1976; File and Deakin 1980; Andrade and Graeff 2001; Andrade et al. 1999), its electrical stimulation induces behavioral and autonomic responses characteristic of the rat’s emotional reaction to conditioned aversive stimuli (Graeff and Silveira Filho 1978).

Recently obtained evidence by Andrade et al. (2004) indicates that serotonergic neurons within the MRN may play a differential role in the regulation of defensive behaviors that have been associated with specific subtypes of anxiety disorders. These authors have demonstrated that in the elevated T-maze test of anxiety (Graeff 1993; Viana et al. 1994; Zangrossi and Graeff 1997; Graeff et al. 1998), whereas the electrolytic lesion of the MRN impaired both inhibitory avoidance and escape behaviors, the selective destruction of MRN 5-HT neurons by the toxin 5,7-DHT only impaired inhibitory avoidance.

In the elevated T-maze, inhibitory avoidance and escape responses are measured in a maze consisting of three elevated arms—one enclosed and two open. For inhibitory avoidance measurement, rats are placed at the distal end of the enclosed arm and the latency to withdraw from this arm with the four paws is recorded in three successive trials. Because of their innate fear of height and openness (Treit et al. 1993), rats learn to remain longer in the enclosed arm over trials, indicating the acquisition of inhibitory avoidance to the open arms. On the other hand, when the animals are placed at the end of one of the open arms, they move towards the closed arm, presumably performing an escape response. Based on the effects of different classes of anxiety-modulating drugs in this test, inhibitory avoidance has been related to generalized anxiety and escape to panic disorder (Graeff 1993; Viana et al. 1994; Zangrossi and Graeff 1997; Graeff et al. 1998; Graeff and Zangrossi 2002; Poltronieri et al. 2003). Therefore, the results of the lesion study performed by Andrade et al. (2004) in the elevated T-maze are suggestive that MRN 5-HT neurons may exert preferential control on defensive behaviors that has been associated with generalized anxiety but not with panic.

In the present study we further addressed the role played by MRN 5-HT neurons in the modulation of the two defensive tasks generated by the elevated T-maze. To this end, we investigated the effects of intra-MRN injection of drugs that either non-selectively or selectively affect the activity of these neurons. In the first group of compounds, muscimol, a GABAA receptor agonist, was used to inhibit MRN neurons, whereas FG 7142, an inverse agonist of benzodiazepine receptors, and kainic acid, an agonist of glutamatergic receptors, were used to stimulate them. Muscimol and FG 7142 affect the activity of 5-HT neurons by respectively enhancing or counteracting tonic GABAergic inhibition (Thiebot et al. 1980; Nishikawa and Scatton 1985; Jones et al. 1986; Shim et al. 1997). The selective manipulation of MRN 5-HT neurons was done by means of drugs that interact with somatodendritic 5-HT1A receptors which control the activity of these neurons by negative feedback (Aghajanian 1972; Kalsner 2000; Beck et al. 2004). Thus, 8-OH-DPAT, a full agonist of 5-HT1A receptors that also interacts with 5-HT7 receptors, was used to inhibit the activity of MRN 5-HT neurons and WAY-100635, an antagonist of 5-HT1A receptors, was used to stimulate it (Hillegaart et al. 1990; Invernizzi et al. 1991; Schreiber and De Vry 1993a,b; Fornal et al. 1996; Dudley et al. 1999).

To investigate the effects of these drugs on locomotor activity, immediately after being tested in the elevated T-maze, animal behavior was also evaluated in an open field.

Material and methods

Animals

Male Wistar rats weighing 200–220 g were housed in groups of five to six per cage until surgery. During the post-surgery period, animals were housed in pairs. Room temperature was maintained at 22±1°C with lights on from 07:00 to 19:00 h. Food and water were freely available throughout the experiment. All procedures were conducted in conformity with the Brazilian Society of Neuroscience and Behavior Guidelines for care and use of laboratory animals, which comply with international laws and policies. All efforts were made to minimize animal suffering.

Apparatus

The elevated T-maze was made of wood and had three arms of equal dimensions (50×12 cm). One arm, enclosed by walls 40 cm high, was perpendicular to two opposed open arms. To prevent falls, the open arms were surrounded by a 1-cm-high Plexiglas rim. The whole apparatus was elevated 50 cm above the floor.

The open-field used to measure locomotion was a wooden square box (60×60 cm) with 30-cm-high walls and with the floor divided into nine squares of 20×20 cm. Luminosity at the level of the maze arms or the open-field center was 50 lx.

Drugs

Kainic acid (Sigma, USA), 8-hydroxy-2-(di-N-propylamino)tetralin (8-OH-DPAT; RBI, USA) and N-{2-[4-(2-methoxyphenyl)-1-piperazinyl]ethyl}-N-(2-pyridinyl)cyclohexanecarboxamide trihydrochloride (WAY-100635; Sigma, USA) were dissolved in 0.9% sterile saline. Muscimol (Sigma, USA) and N-methyl-β-carboline-3-carboxamine (FG 7142; Sigma, USA) were dissolved in a saline–2% Tween 80 solution.

Surgical procedure

After anesthesia with sodium thiopental (50 mg/kg, IP) associated with local anesthesia (2% xylocaine with vasoconstrictor), a 15-mm-long stainless steel guide cannula (0.6 mm of external diameter) was stereotaxically implanted aimed at the MRN (the cannula tip rested 1.5 mm above the MRN) at an angle of 20° with the vertical plane to avoid the sagittal sinus. The following coordinates from bregma were used: posterior=−7.8; lateral=2.9; deep=9.0 mm. The cannula was attached to the skull with acrylic resin and two stainless steel screws. A stylet was introduced into the cannula to prevent obstruction.

Behavioral analysis

On the fifth and sixth days after surgery, animals were gently handled by the experimenter for 5 min. On the sixth day, each animal was pre-exposed for 30 min to one of the open arms of the elevated T-maze. A wood barrier mounted on the border of the maze central area and the open arm’s proximal end isolated this arm from the rest of the maze. It has been shown that this pre-exposure, by shortening latencies to withdrawal from the open arm during the test, renders the escape task more sensitive to the effects of panic-modulating drugs (Teixeira et al 2000; Poltronieri et al 2003).

On the seventh day, rats were randomly assigned to different treatment groups and tested in the elevated T-maze. In experiment 1, animals were injected into the MRN (0.2 μl) with muscimol (0.11 or 0.22 nmol; i.e., 0.0125 or 0.025 μg) or vehicle solution (n=8 per group). In experiment 2, rats were injected with FG 7142 (0.04 or 0.08 nmol; i.e., 9.0×10−3 or 18.0×10−3 μg) or vehicle solution (n=8–10). In experiment 3, animals were microinjected with kainic acid (0.03 or 0.06 nmol; i.e., 6.4×10−3 or 12.8×10−3 μg) or saline (n=10–12). In experiment 4, 8-OH-DPAT (0.6, 3 or 15 nmol; i.e., 0.2, 1.0 or 5.0 μg) or saline (n=10, for each group) was administered. Finally, in experiment 5, rats received an intra-MRN injection of WAY-100635 (0.18, 0.37 or 0.74 nmol; i.e., 0.1, 0.2 or 0.4 μg) or saline (n=10, for each group). For drug injection, a needle (0.3 mm in outer diameter) was introduced through the guide cannula until its tip was 1.5 mm below the cannula end. The drugs were injected over a period of 60 s using a 10-μl microsyringe (Hamilton 701-RN, USA) attached to a microinfusion pump (KD Scientific, USA). The displacement of an air bubble inside the polyethylene catheter connecting the syringe needle to the intracerebral needle was used to monitor the microinjection. The intracerebral needle was removed 60 s after the end of the injection.

Ten minutes after injection, animals were tested in the elevated T-maze. To this end, each animal was placed at the distal end of the enclosed arm of the elevated T-maze facing the intersection of the arms. The time taken by the rat to leave this arm with the four paws (baseline latency) was manually recorded by an observer standing ∼1.5 m far from the test apparatus. The same measurement was repeated in two subsequent trials (avoidance 1 and 2) at 30-s intervals, during which animals were placed in a Plexiglas cage to which they had been habituated. After avoidance measurement (30 s), each animal was placed at the end of the same open arm used in the pre-exposure session, and the time taken to leave this arm with the four paws was recorded in three consecutive trials (escape 1 to 3), again with 30-s intertrial intervals. A cutoff time of 300 s was established for the avoidance and escape latencies.

Immediately after being tested in the elevated T-maze, the animals were individually placed in the open field for the evaluation of locomotor activity and the total number of squares crossed by the animal was recorded over a period of 5 min.

Histology

At the end of the experiments rats were sacrificed under deep 25% urethane anesthesia. The brain was perfused through the heart with 10% formalin before being removed for histological analysis. Brain slices of 55 μm were obtained with a microtome in order to localize the drug injection sites according to the Paxinos and Watson (1998) atlas. Incorrect cannula placement was identified in 25.5% of the animals tested. Only animals with injection sites located inside the MRN were included in the statistical analysis.

Data analysis

A two-way analysis of variance (ANOVA) with repeated measures was used to analyze avoidance and escape data in the elevated T-maze, with treatment as the independent factor and trials (Baseline, Avoidance 1 and 2 or Escape 1, 2 and 3) as the repeated measure. In case of interaction between the independent and the repeated factor, one-way ANOVA, followed by the Duncan post-hoc test, was performed. Locomotor activity in the open field was analyzed by one-way ANOVA followed by the Duncan test.

Results

Figure 1 depicts the sites of drug injections into the MRN of animals tested in present study.

Fig. 1
figure 1

Diagrammatic representation of coronal sections through the rat brain showing the location of injection sites inside (circles) and outside (squares) the MRN. Animals were injected with muscimol (a), FG 7142 (b), kainic acid (c), 8-OH-DPAT (d), WAY-100635 (e) or with their respective control solutions. Figures represent coordinates from Paxinos and Watson (1998) rat brain atlas, with respect to bregma. The number of points in the figures is less than the total number of rats used (n=220) because of several overlaps. (f) Photomicrograph showing a typical injection site in the MRN (indicated by arrow)

Full size image

Experiment 1: effects of muscimol

As shown in Fig. 2 (upper panel), inhibitory avoidance was impaired by treatment with muscimol. Two-way ANOVA showed a significant effect of treatment (F2,21=45.7, P<0.001), trial (F2,42=29.53, P<0.001) and a significant treatment by trial interaction (F4,42=24.12, P<0.001). The Duncan post-hoc test showed a significant difference (P<0.05) between the groups treated with muscimol (0.11 and 0.22 nmol) and the control group at avoidance 1 and 2.

Fig. 2
figure 2

Effects (mean±SEM) of intra-MRN injection of muscimol or vehicle solution on inhibitory avoidance (upper panel) and escape (lower panel) latencies measured in the elevated T-maze. n=8 for each group. *P<0.05 compared with the control group in the same trial

Full size image

The lower panel in Fig. 2 shows that muscimol facilitated escape performance. Two-way ANOVA revealed a significant effect of treatment (F2,21=7.52, P<0.01), but not of trial or a treatment by trial interaction.

Table 1 shows that locomotor activity in the open field was affected by muscimol (F2,21=6.89, P<0.01). Post-hoc analysis revealed that rats treated with the two doses of muscimol crossed more squares (P<0.05) when compared to the control animals.

Table 1 Mean (±SEM) of the number of squares crossed in the open field
Full size table

Experiment 2: effects of FG 7142

As shown in Fig. 3 (upper panel), inhibitory avoidance was facilitated by treatment with FG 7142. Two-way ANOVA showed a significant effect of treatment (F2,25=26.57, P<0.001), trial (F2,50=26.41, P<0.001) and a significant treatment by trial interaction (F4,50=2.60, P<0.05).

Fig. 3
figure 3

Effects (mean±SEM) of intra-MRN injection of FG 7142 or vehicle solution on inhibitory avoidance (upper panel) and escape (lower panel) latencies measured in the elevated T-maze. n=8–10. *P<0.05 compared with the control group in the same trial

Full size image

The lower panel in Fig. 3 shows that FG 7142 impaired escape performance. Two-way ANOVA revealed a significant effect of treatment (F2,25=15.36, P<0.001), trial (F2,50=18.65, P<0.001) and treatment by trial interaction (F4,50=7.32, P<0.001). Post-hoc analysis revealed that animals treated with the two doses of FG 7142 had longer (P<0.05) escape 1 and 2 latencies when compared to the control animals.

One-way ANOVA showed that the effect of FG 7142 in decreasing locomotor activity in the open field was marginal to the statistical significance (F2,25=2.54, P=0.09, see Table 1).

Experiment 3: effects of kainic acid

Figure 4 (upper panel) shows that treatment with kainic acid facilitated acquisition of inhibitory avoidance. Two-way ANOVA revealed a significant effect of treatment (F2,29=23.67, P<0.001), trial (F2,58=10.18, P<0.001) and a significant treatment by trial interaction (F4,58=3.35, P<0.05). Post-hoc analysis showed a significant difference (P<0.05) between animals treated with kainic acid (0.03 and 0.06 nmol) and saline at baseline and avoidance 1.

Fig. 4
figure 4

Effects (mean±SEM) of intra-MRN injection of kainic acid or saline on inhibitory avoidance (upper panel) and escape (lower panel) latencies measured in the elevated T-maze. n=10–12. *P<0.05 compared with the control group in the same trial

Full size image

On the other hand, escape was impaired by kainic acid (Fig. 4, lower panel). Two-way ANOVA revealed a significant effect of treatment (F2,29=8.87, P<0.001), but not of trial or a treatment by trial interaction.

Table 1 shows that the locomotor activity in the open-field was decreased by treatment with kainic acid (F2,29=5.19, P<0.05). The Duncan test revealed that the group treated with 0.06 nmol of kainic acid crossed fewer squares (P<0.05) when compared to the control animals.

Experiment 4: effects of 8-OH-DPAT

As illustrated in Fig. 5 (upper panel), intra-MRN injection of 8-OH-DPAT impaired inhibitory avoidance acquisition. Two-way ANOVA showed a significant effect of treatment (F3,36=9.32, P<0.001), trial (F2,72=25.99, P<0.001) and a significant treatment by trial interaction (F6,72=5.26, P<0.001). The Duncan test showed a significant difference (P<0.05) between the groups treated with the three doses of 8-OH-DPAT and control animals at avoidance 1 and 2.

Fig. 5
figure 5

Effects (mean±SEM) of intra-MRN injection of 8-OH-DPAT or saline on inhibitory avoidance (upper panel) and escape (lower panel) latencies measured in the elevated T-maze. n=10 for each group. *P<0.05 compared with the control group in the same trial

Full size image

Escape from the open arm (Fig. 5, lower panel) and locomotion in the open field (Table 1) were not affected by 8-OH-DPAT.

Experiment 5: effects of WAY-100635

In contrast to 8-OH-DPAT, treatment with WAY-100635 facilitated acquisition of inhibitory avoidance (Fig. 6, upper panel). Two-way ANOVA showed a significant effect of treatment (F3,36=19.86, P<0.001), trial (F2,72=68.22, P<0.001) and a significant treatment by trial interaction (F6,72=2.48, P<0.05). The Duncan post-hoc test showed a significant difference (P<0.05) between the groups treated with WAY-100635 (0.37 and 0.74 nmol) and the control animals at baseline, avoidance 1 and 2.

Fig. 6
figure 6

Effects (mean±SEM) of intra-MRN injection of WAY-100635 or saline on inhibitory avoidance (upper panel) and escape (lower panel) latencies measured in the elevated T-maze. n=10 for each group. *P<0.05 compared with the control group in the same trial

Full size image

Escape from the open arm (Fig. 6, lower panel) and locomotion in the open field were not affected by WAY-100635 (Table 1).

Discussion

In the present study we evaluated the role of MRN 5-HT neurons in the regulation of two defensive behaviors, inhibitory avoidance and one-way escape, of rats submitted to the elevated T-maze test of anxiety. These behaviors have been related to generalized anxiety and panic disorders, respectively (Graeff et al. 1993; Viana et al. 1994; Zangrossi and Graeff 1997; Graeff and Zangrossi 2002).

The results showed that these two defensive tasks were differentially affected by drugs that non-selectively inhibit or stimulate the activity of serotonergic neurons in the MRN. Thus, microinjection of FG 7142 and kainic acid, drugs that stimulate these cells (Thiebot et al. 1980; Goodchild et al. 1982; Jones et al. 1986; Tao et al. 1997), facilitated inhibitory avoidance acquisition while at the same time impairing escape performance. On the other hand, intra-MRN injection of muscimol, a drug that inhibits the activity of serotonergic neurons (Nishikawa and Scatton 1985; Shim et al. 1997), impaired the acquisition of inhibitory avoidance and facilitated escape performance.

Regarding the effects of drugs that selectively interfere with the activity of MRN serotonergic neurons, our results showed that while the 5-HT1A receptor agonist 8-OH-DPAT impaired inhibitory avoidance, the antagonist of the same receptors WAY-100635 facilitated the acquisition of this defensive task. The two drugs were ineffective in changing escape performance.

Previous studies have consistently demonstrated that while intra-MRN administration of muscimol causes hyperlocomotion (Wirtshafter et al. 1987; Wirtshafter and Klitenick 1989; Shim et al. 1997), microinjection of kainic acid has the opposite effect (Wirtshafter and McWilliams 1987). In agreement, we presently found that locomotion in the open-field was increased after muscimol and decreased after kainic acid. These results indicate that the stimulatory motor effect of the former compound and the inhibitory effect of the latter have contributed, respectively, to the observed shortening and lengthening of latencies in the elevated T-maze. Change in the motor activity of the animals also seem to account for the effect of FG 7142 in the elevated T-maze, since this drug prolonged avoidance and escape latencies but also tended to decrease (P=0.09) locomotion in the open field.

In contrast to drugs that modulate GABA and glutamate neurotransmission in the MRN, various studies have indicated that the selective pharmacological manipulation of serotonergic neurons in this brainstem area exerts a smaller influence on locomotor activity (for a review, see Shim et al. 1997). Accordingly, our study showed that neither 8-OH-DPAT nor WAY-100635 altered rat exploration in the open field test.

To our knowledge, no other study had addressed the effects of intra-MRN injection of muscimol, FG 7142 and kainic acid on anxiety or fear-related behaviors. A different picture emerges when the effects of drugs acting on 5-HT are considered. Thus, in agreement with the effect of 8-OH-DPAT on the elevated T-maze inhibitory avoidance task, it has been shown that intra-MRN injection of this agonist reduces the expression of anxiety-related behaviors in various animal models such as the elevated plus-maze (File et al. 1996; De Almeida et al. 1998), the social interaction test (File et al. 1996), the black–white transition test, the conditioned suppression of drinking (Carli and Samanin 1988), and contextual fear conditioning (Avanzi and Brandrão2001). As also observed in the present analysis, this anxiolytic effect was achieved at doses that do not consistently change the general motor activity of the animals.

The most important finding of the present study is the observation that 8-OH-DPAT causes an anxiolytic effect exclusively on the inhibitory avoidance task of the elevated T-maze. This result is in accordance with a previous finding from this laboratory showing that the selective lesion of MRN 5-HT neurons by the neurotoxin 5,7-DHT caused a similar effect. Interestingly, the electrolytic lesion of the same nucleus, which non-specifically destroys neurons and fibers of passage, changed both inhibitory avoidance and escape performance (Andrade et al. 2004). Taken together, these results suggest that, in the elevated T-maze test of anxiety, MRN 5-HT neurons are selectively involved in the regulation of inhibitory avoidance behavior.

The effect of intra-MRN injection of WAY-100635 on the elevated T-maze further indicates that MRN 5-HT neurons are selectively involved in the modulation of inhibitory avoidance. As demonstrated in experiment 5, the administration of this 5-HT1A receptor antagonist facilitated inhibitory avoidance, indicating an anxiogenic effect, without changing escape.

The MRN sends 5-HT projections to many anxiety and fear-related areas such as the amygdala, hippocampus and the frontal cortex (Vertes et al. 1999). These areas, especially the first two, have been implicated in the mediation of defensive behaviors that are displayed under conditions of potential threat which involves an approach avoidance conflict situation (for a review, see Gray and McNaughton 2000; Graeff 2002). Activation of 5-HT1A receptors in these post-synaptic areas has been shown to facilitate the expression of these behaviors (Hodges et al. 1987; Andrews et al. 1994; File et al. 1996; Hamon 1997; Romaniuk et al. 2001). In this respect, the anxiolytic effect of 8-OH-DPAT observed after its injection into the MRN could be interpreted in terms of a decrease of 5-HT release in the hippocampus and/or the amygdala resulting from the interaction of this drug with inhibitory somatodendritic 5-HT1A autoreceptors. This hypothesis is supported by numerous microdialysis and electrophysiological studies (Sharp et al. 1990; Invernizzi et al. 1991; Adell et al. 1993; Kreiss and Lucki 1994; McQuade and Sharp 1997). Along the same line of thinking, WAY-100635 in the MRN increases anxiety by preventing a tonic inhibitory influence of endogenous 5-HT acting on the same somatodendritic 5-HT1A autoreceptors. This would result in an increase in the firing rate of the 5-HT neurons and consequently in an increase of 5-HT release in postsynaptic areas. Although WAY-100635 was originally considered a silent antagonist (Fletcher et al. 1996), a growing body of evidence has demonstrated that this drug can, in fact, increase the firing rate of 5-HT neurons (Mundey et al. 1996; Fornal et al. 1996; Bjorvatn et al. 2000; Hajós et al. 2001) and increase the release of 5-HT in projection areas of both the median and the dorsal raphe nuclei (Bosker et al. 1996; Dudley et al. 1999).

In conclusion, the results of the present study demonstrate that MRN serotonergic neurons are selectively involved in the modulation of inhibitory avoidance in the elevated T-maze. These results support the proposal that these cells play an important role in anxiety processing (Gray and McNaughton 2000), with implication for pathologies such as generalized anxiety disorder.