Episodic activity in a heterogeneous excitatory network, from spiking neurons to mean field

Abstract

Many developing neural systems exhibit spontaneous activity (O’Donovan, Curr Opin Neurobiol 9:94–104, 1999; Feller, Neuron 22:653–656, 1999) characterized by episodes of discharge (active phases) when many cells are firing, separated by silent phases during which few cells fire. Various models exhibit features of episodic behavior by means of recurrent excitation for supporting an episode and slow activity-dependent depression for terminating one. The basic mechanism has been analyzed using mean-field, firing-rate models. Firing-rate models are typically formulated ad hoc, not derived from a spiking network description, and the effects of substantial heterogeneity amongst the units are not usually considered. Here we develop an excitatory network of spiking neurons (N-cell model) with slow synaptic depression to model episodic rhythmogenesis. This N-cell model displays episodic behavior over a range of heterogeneity in bias currents. Important features of the episodic behavior include orderly recruitment of individual cells during silent phases and existence of a dynamical process whereby a small critical subpopulation of intermediate excitability conveys synaptic drive from active to silent cells. We also derive a general self-consistency equation for synaptic drive that includes cell heterogeneity explicitly. We use this mean-field description to expose the dynamical bistability that underlies episodic behavior in the heterogeneous network. In a systematic numerical study we find that the robustness of the episodic behavior improves with increasing heterogeneity. Furthermore, the heterogeneity of depression variables (imparted by the heterogeneity in cellular firing thresholds) plays an important role in this improvement: it renders the network episodic behavior more robust to variations in excitability than if depression is uniformized. We also investigate the effects of noise vs. heterogeneity on the robustness of episodic behavior, especially important for the developing nervous system. We demonstrate that noise-induced episodes are very fragile, whereas heterogeneity-produced episodic rhythm is robust.

1 Introduction

Many developing neural systems (retina, hippocampus, spinal cord) exhibit spontaneous activity (O’Donovan 1999; Feller 1999) that is characterized by episodes of discharge (active phases) when many cells are firing, separated by silent phases during which few cells fire (cf. Fig. 1 for the developing spinal cord). This spontaneous activity is thought to affect the development of neuronal circuits (Katz and Shatz 1996; Ben-Ari 2001; Stellwagen and Shatz 2002; Borodinsky et al. 2004; Hanson and Landmesser 2004). Spontaneous activity in the form of slow oscillations between upstates and downstates (Timofeev et al. 2000) or random brief events (Beggs and Plenz 2003) has also been studied in vitro, beyond the context of development.

Fig. 1

Spontaneous episodes of synaptic drive (population-averaged, low pass-filtered membrane potential) recorded from the ventral roots of chick embryo spinal cord in vitro at embryonic day 7.5. Note the high degree of population synchrony during the active phase: many cells in the network are strongly depolarized. Spike-to-spike, however, the network is asynchronous on the membrane time scale. Silent phases can be several minutes in duration. Modified from Tabak et al. (2000)

Several models have shown that episodic activity could be generated through a regenerative mechanism whereby fast recurrent excitation recruits the network and activity-dependent depression terminates the episodes. This depression can be synaptic (Tabak et al. 2000; Tsodyks et al. 2000; Loebel and Tsodyks 2002; Wiedemann and Luthi 2003) or cellular, such as spike-frequency adaptation (Latham et al. 2000; Timofeev et al. 2000; Wiedemann and Luthi 2003). It can accumulate rapidly during spiking so active phases are brief and pulsatile (Tsodyks et al. 2000; Loebel and Tsodyks 2002; Wiedemann and Luthi 2003) or accumulate slowly to give broader active phases (Latham et al. 2000; Tabak et al. 2000; Timofeev et al. 2000), as we will consider here. Formulations have involved networks of many spiking units or mean-field (spatiotemporally averaged) firing rate descriptions; the latter are not typically derived explicitly from spiking-based details.

For our case study, we consider the developing chick spinal cord. As in many developing circuits, GABAergic and glycinergic neurotransmission is functionally excitatory since the synapses are depolarizing (Cherubini et al. 1991; Sernagor et al. 1995; Chub and O’Donovan 1998). In the spinal cord, an activity-dependent depression of network excitability (Fedirchuk et al. 1999; Tabak et al. 2001) appears to underlie the episodic activity. The interepisode interval (silent phase) is then a period of recovery from this depression. Previously, using an ad hoc mean-field model, we have accounted for various aspects of episodic rhythmogenesis (Tabak et al. 2000). That excitatory network model was bistable on a fast time scale and showed abrupt switching between active and silent phases; slow synaptic depression mediated the episodic rhythm. The behavior was robust to changes in connectivity, but not in cell excitability. Heterogeneity in the spiking network of the current paper remedies this sensitivity.

Here, we introduce an excitatory network model of spiking (leaky integrate-and-fire) neurons with slow synaptic depression (we call it the N-cell model) and investigate its episodic rhythmogenic properties as follows. First, we show with numerical simulations that the noise-free N-cell model generates episodic activity for various parameter settings and heterogeneous I-distributions with only a fraction (possibly small) of intrinsically spiking cells. A consequence of heterogeneity that was first described by Tsodyks et al. (2000) and confirmed by Wiedemann and Luthi (2003) is the existence of a subpopulation of intermediate excitability that is especially important for episodic behavior. We extend those findings by studying how the degree of heterogeneity affects this and other subpopulations critical to episodic rhythmogenesis.

Second, we derive directly from the N-cell spiking model a new mean-field description that includes heterogeneity explicitly. It accounts for the pseudosteady state firing during the active and silent phases, given a distribution of depression variables. Using this description we demonstrate that the mean field with uniform synaptic depression does not fully capture the observed behavior; for instance, the network can be rhythmic when the mean-field description is not bistable. Third, using our mean-field formulation with simulation-produced synaptic depression variables at each moment of the rhythmic cycle, we demonstrate that the heterogeneous distribution of slow depression variables renders the network dynamically bistable and underlies episodic behavior.

We then obtain a central result of this paper in a systematic numerical study: we define robustness (dynamic range) as a region in the parameter space supporting rhythmicity and find that robustness improves with increasing heterogeneity. Furthermore, the heterogeneity of depression variables (imparted by the heterogeneity in cellular firing thresholds) plays an important role in this improvement: it renders the network episodic behavior more robust to variations in excitability than if depression is uniformized.

Finally, another way to randomize neuronal spiking and promote episodic behavior may be to add noisy inputs. We thus examine whether episodic behavior would benefit from the addition of stochasticity—instead of heterogeneity. We find that the resulting episodic behavior is extremely sensitive to variations in cellular excitability. Therefore, our results suggest that heterogeneity is a determinant factor for excitatory networks to generate robust episodic activity. This may be relevant to immature networks for which normal development is thought to require a robust pattern of spontaneous activity.

2 Methods

2.1 Equations of the ad hoc model

Based on the experimental findings described in Introduction, Tabak et al. (2000) introduced an ad hoc firing-rate model (1–3) meant to describe the (spatiotemporal) average activity a in the network and possessing the following essential features. Recurrent excitatory connectivity (n measures its strength) provided positive feedback modeled by the increasing sigmoidal function a _∞. As is typical in such firing-rate models (Wilson and Cowan 1972), one assumed that a _∞(0) > 0, i.e., that some (small) fraction of cells were intrinsically spiking (or, that there was a nonzero probability of neurotransmitter release). This assumption led to network bistability (low and high steady states of the activity variable a). A slow activity-dependent (s _∞ was a decreasing sigmoidal function of a) synaptic depression variable s was used to induce transitions between the high and low activity states, hence the episodic behavior. The variable d corresponded to faster depression, responsible for cycling within each episode, which we do not model here.

$$\tau_a\dot{a} = a_\infty(n \cdot s \cdot d \cdot a) - a$$

(1)

$$\tau_d\dot{d} = d_\infty(a) - d$$

(2)

$$\tau_s\dot{s} = s_\infty(a) - s$$

(3)

2.2 N-cell model formulation

We introduce a cell-based, spiking network model with the following salient properties:

• 1. N leaky integrate-and-fire neurons.

• 2. The network is heterogeneous: cells have effectively different firing thresholds due to the heterogeneity of bias currents I _i . We use uniform distributions here. A fraction of the cells are assumed to be intrinsically spiking (fire tonically when isolated).

• 3. Fast, purely excitatory synapses (GABA and glycine are functionally excitatory: Cherubini et al. 1991; Sernagor et al 1995).

• 4. Slow synaptic depression modulates fast synaptic activation. We do not necessarily distinguish between presynaptic and postsynaptic depression sites, but the depression is due to presynaptic activity (Fedirchuk et al. 1999).

The model equations are:

$$\tau \dot{V_i} = -V_i+I_i-(V_i-V_{\mathit{syn}})g_{\mathit{syn}},\: i=1 \ldots N$$

(4)

$$g_{\mathit{syn}} = {\overline{g}}_{syn} N^{-1}\sum_{j=1}^N q_j s_j$$

(5)

$$V_i(t_f) = 1 \Longrightarrow V_i(t)=0 \mbox{ for } t_f < t \leq t_f+\tau_\mathit{ref}$$

$$\dot{q}_i = \alpha_q(t)(1-q_i)-\beta_q q_i$$

(6)

$$\dot{s}_i = \alpha_s(1-s_i)-\beta_s(t)s_i$$

(7)

Here V _i is the membrane potential of neuron i, and g _syn is the total (macroscopic) synaptic field (drive) produced by the combined recurrent activity of all cells in the network and normalized by N. This limits the maximum value of g _syn to ${\overline{g}}_{syn}$, the maximum synaptic conductance, independent of the network size. q _i is the fast synaptic conductance activation of neuron i, or, more precisely, fraction of synaptic connections (relative to the maximum) made available due to neuron i firing on the presynaptic side. In the absence of depression, q _i is the fraction of open postsynaptic channels in neurons for which neuron i is presynaptic. s _i is the slow depression, i.e., the fraction of nondepressed efferent synapses of cell i. s _i modulates synaptic activation q _i and can represent the availability of resources or effectiveness of transmission. For example, a synaptic connection can become depressed (not available for transmission) via activity-dependent depletion of the neurotransmitter on the presynaptic side, or via receptor desensitization on the postsynaptic side. Yet alternatively, it can represent a mechanism similar to retrograde inhibition, which acts on the presynaptic side, but under the influence of a chemical substance released from the postsynaptic side in response to presynaptic activity. We do not specifically distinguish among all these possibilities in order to keep our approach general.

We assume that the synaptic current to each neuron flows through such ionic channels that can be combined into one equivalent channel with a linear I–V relationship.

This model is dimensionless with respect to all quantities other than time. Specifically, if G _syn and g _L are the actual maximum synaptic and leakage conductances, respectively, then ${\overline{g}}_{syn}=G_{syn}/g_L$ is the maximum synaptic conductance relative to leakage. Now, let v be the actual membrane potential and v _rest , v _th , and v _syn the actual rest, threshold, and synaptic reversal potentials, respectively. Then V = (v − v _rest )/(v _th  − v _rest ), which makes the membrane potential normalized so that the rest potential is equal to 0, and the firing threshold is equal to 1. Similarly, V _syn  = (v _syn  − v _rest )/(v _th  − v _rest ), and if i _app is the actual bias current, then I = i _app  / (g _L (v _th  − v _rest )). Finally, τ = C/g _L , where C is the actual neuronal membrane capacitance.

In this work, we use uniform bias current distributions of varying degree of heterogeneity, with γ denoting the fraction of intrinsically spiking cells in the network, I _min denoting the minimum, and 〈I〉, the average bias current of the distribution. However, both the N-cell model and the mean-field description derived later in this paper are valid for an arbitrary distribution. I = 1 represents the normalized firing threshold in terms of the bias current, i.e., cells with I < 1 are excitable (do not fire when isolated), and cells with I > 1 are intrinsically spiking (fire tonically when isolated). We define large heterogeneity to denote distributions with the spread of bias currents (width of the distribution) equal to 1, and reduced heterogeneity to indicate that the spread is 0.4.

The equation for the fraction of open channels q _i is derived from a standard first-order argument with α _q (t) and β _q being the transition probabilities from the closed to the open state and from the open to the closed state of a gated channel, respectively. The function α _q (t) is set equal to a constant α _q during the time period of short duration ε _q after each spike. Slow depression s _i represents a slow-acting negative feedback mechanism that affects the effectiveness of synaptic transmission. In our model, slow negative feedback is governed by a law similar to that for q _i , but with the rate of depression β _s (t) set equal to a constant β _s during the time period of short duration ε _s after each spike.

Parameter values or their ranges used in this work are given in Table 1. Specific values within the ranges are stated in the text as appropriate. Here we emphasize that both the individual membrane potentials and synaptic activations are fast compared to slow depression variables. For V _syn , a value significantly above 1 is qualitatively comparable to values observed during earlier stages of development.

Table 1 Parameter values for the N-cell model

2.3 N-cell model implementation and numerical solution

The N-cell model was implemented in a software package developed in C++ by Boris Vladimirski.

The numerical integration of the system was performed via a two-step, second-order Runge-Kutta method, also known as modified Euler’s method (Burden and Faires 2001, pp. 272–277). However, since the right-hand sides of (6) and (7) are discontinuous at each spike, we must use spike-time interpolation (Hansel et al. 1998; Shelley and Tao 2001) to retain the second-order accuracy (cf. Vladimirski 2005, for details).

The time step used was typically 0.001τ = 0.01/α _q to guarantee that no significant features could be missed while integrating over a longer time period. For Figs. 5 and 9 the time step was equal to 0.01τ.

Data produced by the numerical simulation software package were processed and the results visualized in Matlab as 2D- and 3D-graphs and animations.

3 Results

3.1 Numerical study of episodic behavior in the heterogeneous N-cell model

We begin by exploring the features of episodic behavior exhibited by the N-cell model (4–7). To describe the network dynamics we show the time courses of g _syn and of the representative samples of depression variables s _i from simulations of the full network (3N dynamic variables). The synaptic drive g _syn is produced by the recurrent excitatory coupling (all-to-all) and is delivered to all cells. It thus provides a measure of the macroscopic network activity and mirrors the instantaneous population mean firing rate. Network activity is organized into active phases, or episodes, during which all cells are spiking, and silent phases, during which only a fraction of cells are spiking. We illustrate the general features of episodic behavior and effects of heterogeneity and excitability by using four specific uniform distributions of bias currents: “large” and “reduced” heterogeneity with bias currents that span an interval of width 1 or 0.4, respectively. These distributions are centered so that just a few cells (10%) or 50% of the cells are intrinsically spiking; γ denotes the fraction of intrinsically spiking cells. The threshold current for spontaneous firing is I = 1 in our formulation.

3.1.1 Basic features of episodic rhythm

With a broad distribution of I _i (large heterogeneity) and just a few intrinsically spiking cells, the silent phases are long compared to the active phases, as typical of experimental data [cf. Figs. 1 and 2(a)]. The overall behavior can be seen via the time courses of g _syn (black) and the population mean of s _i , 〈s〉 (green). During the active phase g _syn decreases slowly, as the synapses depress (〈s〉 decreasing). Eventually, g _syn reaches a critical level and the network can no longer sustain the high activity state; g _syn drops precipitously, ending the active phase. During the silent phase g _syn is small and relatively flat with modest increase as the synapses recover from depression (〈s〉 increasing). Eventually the slowly growing g _syn becomes sufficiently large and the synapses of the silent cells recover enough for the regenerative effect of mutual excitation to lead to the final stage of rapid recruitment of all cells into a next episode. (This final stage can be prompted by fluctuations in g _syn caused by finite-size effects.) Thereafter, the sequence of active and silent phases continues periodically.

Fig. 2

Episodic behavior for large I-heterogeneity. The time courses of synaptic drive g _syn (black) and average slow depression 〈s〉 (green) are shown on the left. Enlargement of the dashed region and a sample of 11 individual slow depressions s _i (solid colored curves) uniformly spaced over the distribution of I (direction of I-increase is indicated by the arrow) are shown on the right. Thick red portions of depression time courses indicate that the corresponding cell is firing; thin blue lines indicate quiescence. Above each row of panels, the I-distribution is shown schematically as a rectangular diagram. I increases from left to right along the diagram, and the scale is the same for both diagrams. The blue (middle) rectangle represents the excitable cells (I _min  ≤ I < 1) and the red (rightmost) rectangle represents the intrinsically spiking ones (I > 1). The white (leftmost) rectangle is shown to align (for convenience) the left boundaries of all the I-diagrams and represents the interval 0 < I < I _min which is not part of the distribution. The height of the diagram is arbitrary. For panels (a) and (b), I _i are uniformly distributed on (0.1; 1.1) with the fraction of intrinsically spiking cells γ = 0.1. For panels (c) and (d), I _i are uniformly distributed on (0.5; 1.5) with $\gamma\!=\!0.5.\,{\overline{g}}_{syn}\!=\!2.0$

A closer look at the dynamics of the system [Fig. 2(b)] shows the wide spread in the s _i -values across the population. The 11 colored solid curves correspond to cells whose I-values are equispaced across the distribution; thick red indicates that the cell is firing, and thin blue, silent. The s _i -values maintain an ordering throughout silent and active phases: cells with higher I _i -values fire more and have lower s _i -values since their synapses suffer greater depression. The spread of s _i -values is smallest when the silent phase begins and all the cells are strongly depressed after the previous active phase. The only cells that are firing at this moment are the intrinsically spiking ones and the fraction of cells that are pushed above the threshold by the intrinsically spiking cells at all times [Fig. 3(a)]. The individual depression variables of the non-firing cells begin to recover at the slow rate α _s toward the non-depressed value of 1 [blue lines in Fig. 2(b)]. The s _i -values for currently firing cells are also recovering [red lines in Fig. 2(b)], but to a smaller value and at a slower rate (since their firing, albeit slower than in the active phase, still causes some depression). Hence, the spread of s _i -values steadily increases during the silent phase and the s _i -values can be classi fied into three groups: higher values for those cells that fire during the active phase only, much lower values for those few cells that fire throughout both phases (cells that are intrinsically spiking and the excitable cells that are sufficiently close to the threshold to be recruited by the intrinsically spiking cells at all times), and intermediate ones for cells that are recruited to fire during the silent phase. These latter cells [indicated by the dashed lines representing non-monotone depression time courses in Fig. 2(b) and dashed orange lines in Figs. 3(a–b)] comprise what we call the intermediary excitable subpopulation. They relay the synaptic drive from the active to the silent cells.

Fig. 3

Episodic behavior for large I-heterogeneity: raster plots. Cells’ bias currents are shown along the y-axis. Black dots indicate that the corresponding cells are firing; white dots indicate quiescence. Top and bottom panels correspond to top and bottom panels, respectively, of Fig. 2. (a) and (d) show several episodes. Note orderly slow recruitment during the silent phase proceeding from the more to less excitable neurons. (b) and (e) are enlargements that demonstrate the fast orderly recruitment at the transition from the silent phase to the active phase. (c) and (f) are enlargements that demonstrate the fast orderly derecruitment at the transition from the active phase to the silent phase. Much larger fraction of cells is active at the beginning of the silent phase in the bottom panels because the network is more excitable. However, the fraction of the cells recruited during the silent phase, which form the intermediary subpopulation (shown by the dashed orange lines), is approximately the same in both panels (5–10%)

The recruitment of the intermediary subpopulation is due to the recovery of firing cells from depression resulting in the gradual growth of the synaptic drive. Both this recruitment [Fig. 3(a)] and the following sudden recruitment of the remaining silent cells [2401 ≤ t/τ ≤ 2406 in Fig. 3(b)] are orderly, as more excitable cells are closer to the firing threshold and are recruited sooner than less excitable ones. During the active phase, all cells fire rapidly and their depression variables decay. Note that, since the decay rate β _s of s _i is larger than the growth rate α _s , the slow exponential-like decrease of s _i (and corresponding decrease in g _syn ) is steeper than the recovery time course of non-firing cells in the silent phase. When g _syn decreases enough, neurons stop firing abruptly, doing so, again, in an orderly sequence, beginning with the least excitable cell [2469 ≤ t/τ ≤ 2475 in Fig. 3(c)].

3.1.2 Characteristic effects of heterogeneity and excitability

To explore the range of possible episodic behaviors and demonstrate that episodic activity is robust with respect to cellular excitability, we simulated the network after translating the I-distribution, centering it at I = 1. Now, with 50% of cells intrinsically spiking, the network becomes more excitable [Fig. 2(c–d); note the difference in time- and g _syn -scales between (a–b) and (c–d)]: there are more thick red curves in Fig. 2(d) as more cells are firing during the silent phase [cf. Fig. 3(d)]. The overall level of firing, compared with the preceding case (10% of cells intrinsically spiking), leads to synapses that are more depressed [lower s _i -values in Fig. 2(d)]. The overall patterns of orderly recruitment during the silent phase [Fig. 3(e)] and derecruitment [Fig. 3(f)] at the end of the active phase are similar to those described in the previous paragraph. In particular, despite much higher network excitability, the percentage of cells recruited during the silent phase (the intermediary subpopulation) remains the same as above at about 5–10% [Fig. 3(d)].

The silent phase is much shorter (the active phase duration is slightly reduced, too) and the amplitude of the g _syn -variations is smaller in Fig. 2(c–d) compared to Fig. 2(a–b). To explain these observations, we first note that the silent phase ends when the intermediary subpopulation has been recruited. Since the fraction of cells comprising this subpopulation is approximately the same between panels (a–b) and (c–d), g _syn has to increase by approximately the same amount during the silent phase. To achieve this increase, the synapses of the active cells in panels (c–d) have to recover much less compared to panels (a–b) because many more cells fire and hence contribute to the synaptic drive. As a result, the range of variation of the s _i -values of the active cells is significantly decreased and so is the time spent in the silent phase in panels (c–d).

In the active phase, the range of variation of the s _i -values of all cells is the same as it is in the silent phase since the episodic behavior is periodic. The difference with the silent phase is that excitable cells contribute strongly to the synaptic drive and hence to the duration of the active phase. In particular, the active phase terminates when the least excitable cell stops firing [Fig. 3(c and f)], and thus the g _syn -level at which the active phase ends is much lower and the range of s _i -variation is both reduced and shifted downward for the more excitable network. The former effect tends to shorten the active phase, whereas the latter one makes it longer: the depression rate is slower during the active phase because all the cells are already more strongly depressed at the end of the silent phase than their counterparts in the less excitable network. As a result, the duration of the active phase is similar between panels (a–b) and (c–d).

To summarize, the observed changes in the duration of both phases as the network excitability is increased are due to the range of s _i -values being shifted downward and reduced. These two effects combine to shorten the silent phase, but oppose each other during the active phase. Interestingly, even though the ad hoc model (Tabak et al. 2000) cannot account for the aspects of episodic rhythmogenesis in the N-cell model considered later, it also predicts (Tabak et al. 2006) that increasing network excitability shortens the silent phase without significantly affecting the active phase for a similar reason (downward shift and contraction of the range of variation of the depression variable).

We now ask how reduced heterogeneity combined with changes in overall cellular excitability affects episodic behavior. For the results in Fig. 4, the spread of the I-distribution is reduced from 1.0 (Fig. 2) to 0.4. With 10% intrinsically spiking cells [Fig. 4(a–b)], the network is still rhythmic but the s _i -values are more homogeneously distributed [compared with Fig. 2(a–b)].

Fig. 4

Episodic behavior for reduced I-heterogeneity. Variables analogous to those in Fig. 2 are plotted using the same colors/linestyles. For panels (a) and (b), I-values are uniformly distributed on (0.64; 1.04) with γ = 0.1. For panel (c), I-values are uniformly distributed on (0.8; 1.2) with γ = 0.5. Other parameters are the same as in Fig. 2. The value of γ in panel (a–b) is the same as that in Fig. 2(a–b), but the s _i -values are more homogeneously distributed. In panel (c), the network excitability is too high, and all cells fire at all times

Although the fraction γ of intrinsically active cells is the same as in Fig. 2(a–b), the silent phase is much shorter and the active phase much longer. This is because the distribution of I is more compact, hence to achieve the same γ the average cell excitability has to be increased. This leads to a short silent phase, similarly to the case of Fig. 2(c–d). The least excitable cell is now much more excitable, and it becomes significantly more difficult to terminate an episode since the s _i -variables have to depress even more. If the minimum I-value is increased further [by translating the reduced heterogeneity distribution upward in I: Fig. 4(c)], the network never leaves the active phase. If, however, we were to reduce ${\overline{g}}_{syn}$ (e.g., to ${\overline{g}}_{syn}=0.75$), episodes would occur, similar to those in Fig. 2(c–d) (results not shown).

To summarize, we have shown here that adequate heterogeneity allows episodic behavior in our N-cell network model of spiking cells. Increasing heterogeneity promotes episodic behavior by providing intrinsically active cells—which are necessary to start an episode—without excessively raising the overall excitability of the network, which would prevent episode termination. We pursue this property of episodic behavior robustness in a later subsection with systematic and extensive variations in the parameters: 〈I〉 and ${\overline{g}}_{syn}$.

3.1.3 Critical and intermediary subpopulations

At this point in the paper, the dynamics of the system might appear to be describable by a mean-field model with one depression variable, e.g., average depression 〈s〉 as in Figs. 2(a and c), and 4(a). However, the more detailed presentation of individual depression variables s _i , particularly in the case of large heterogeneity in Fig. 2(b) and (d), demonstrates the heterogeneity of the s _i -distribution as it separates into three distinct groups. One important role of this heterogeneity in episodic rhythmogenesis was demonstrated by Tsodyks et al. (2000) and Wiedemann and Luthi (2003): a small group of cells (we call this group the intermediary subpopulation) with intermediate excitability was always recruited before the next population burst occurred. Furthermore, this intermediary subpopulation could not be removed without disrupting population bursts. Here we systematically study the properties of the intermediary subpopulation and critical subpopulations (the latter are contiguous groups of cells such that their removal jeopardizes the episodic rhythm) for various levels of heterogeneity in the I-distribution.

The consecutive orderly recruitment of cells during the silent phase is analogous to the propagation of an excitation wave, except here the neighbors are in the I-dimension, not spatial dimension. When a cell starts firing, it contributes to the total g _syn which then stimulates the other cells, in particular, beckons the cell that is next nearest to its effective firing threshold. The process is not linear: the contributions are not the same by successive recruitees since they enter the active group with different degrees of depression in their efferent synapses. Our raster plots (Fig. 3) illustrate the evolution of the recruitment wave. We observe that there is an intermediary subpopulation of cells (5–10% of the whole network whose I _i -values are just below those of the excitable cells recruited by the intrinsically spiking cells at all times; indicated by the dashed orange lines in Fig. 5) that, once recruited and provided that the silent cells have sufficiently recovered, quickly triggers all the remaining silent cells and the next active phase proceeds.

Consider the I-distribution in Fig. 2(c–d). If we remove those 10% of cells just below I = 1 (with 0.9 < I < 1.0) and redistribute them uniformly into the subpopulation with I < 0.9, episodes persist, and their quantitative properties stay essentially the same. This finding is consistent with Tsodyks et al. (2000). However, here we also try a new manipulation: if instead we remove the 10% of cells with 0.7 < I < 0.8 , which include all the cells recruited during the silent phase, and redistribute these uniformly into the subpopulation with I > 0.8, episodes disappear. Surprisingly, even though the network is biased to have a larger fraction of more excitable and intrinsically spiking cells, these cells are unable to recruit the less excitable cells that are below the gap between 0.8 and 0.7. The safety factor for the recruitment wave has been reduced too much, recruitment fails and episodic rhythmicity is jeopardized. This indicates that the intermediary population is indeed critical and highlights the importance of heterogeneity in the depression variables to the successful recruitment.

Fig. 5

Recruitment-critical subpopulations for I-distributions from Figs. 2(a)–4(b). Each rectangle with a solid black edge superimposed on a distribution diagram represents a critical subpopulation (of same width and position) such that if it is removed, the episodic behavior disappears. The absence of such a rectangle around a particular position on the diagram indicates that this position is not recruitment-critical for the given width. The widths of all the rectangles for each distribution are identical, chosen to be the minimum values of 0.01, 0.01, and 0.0432 for the top, center, and bottom diagrams, respectively, that produce any critical subpopulations (the heights are arbitrary to separate individual rectangles). Dashed orange rectangles approximately represent all the cells recruited during the silent phase (5–10% of the whole network)

We also ask if there are any other critical subpopulations, perhaps not necessarily part of the intermediary one. To that end, we systematically vary the position of a small continuous subpopulation along the entire range of each distribution and plot a black rectangle of the same width at the corresponding location in Fig. 5 if this subpopulation is critical. To expose the most sensitive areas, we look at the minimum widths that reliably produce critical subpopulations. For the top two diagrams, the very small width of 0.01 = 1% of the network still produces critical subpopulations, whereas for the distribution in the bottom diagram, only 0.0432 = 10.85% does, but 0.04 = 10% does not. If a distribution is more excitable and/or homogeneous, it is easier for the active cells to bridge a gap of a given size. For example, compared to the top diagram, the critical subpopulations lie among the less excitable cells in the center diagram and the number of such subpopulations is smaller, whereas in the bottom diagram, the size of each critical subpopulation is much larger.

Comparing the black and orange rectangles in Fig. 5, we conclude that there can be several critical subpopulations of minimal size for each I-distribution. Furthermore, while the intermediary subpopulation is always critical (i.e., contains at least one critical subpopulation of minimum size), there are critical subpopulations that are not part of the intermediary one.

3.2 Mean-field description

In this section, we derive a reduced analytical mean-field description (cf. also Ermentrout 1994; Shriki et al. 2003) for g _syn based on the separation of the time scales between fast membrane potentials and synaptic activation s (we call their time scale fast) and slow synaptic depression variables (we call their time scale slow). Our mean-field description consists of N slow depression variables coupled with one nonlinear equation for g _syn and is derived by averaging over the cells’ asynchronous spiking behavior. The separation of the time scales is taken to an extreme so that at each moment in time the fast variables are assumed to be at their pseudosteady states. We talk about pseudosteady states here because within this mean-field description, at each moment in slow time the fast variables are allowed to equilibrate to their steady states (while the slow depression variables are treated as parameters), but these steady states are functions of the depression variables which change on the slow time scale only. This step is a key part of a full fast-slow dissection of the dynamics and it should reveal bistability between high activity and low activity states of the network. Having this pseudosteady state approximation for the network’s behavior, we then overlay uniform and heterogeneous slow depression dynamics to see how they sweep the system back and forth between the active and silent phases (as done for our earlier ad hoc model).

The mean-field description derivation is based on the following assumptions:

• 1. The synaptic input to each neuron is the same for all neurons, i.e., we assume that the network is fully-connected and there is no stochastic release. The full connectivity requirement can be lifted for a sparsely connected system in which the probability of a synaptic connection between any two neurons in the network follows the same probability distribution. Similarly, if stochastic release follows the same rule for all neurons, it can be incorporated into the derivation easily.

• 2. Each neuron has entered a periodic regime and is firing at a constant frequency (again, on the fast time scale), i.e., all the transients have disappeared.

• 3. The synaptic input to each neuron is constant in (fast) time. This is reasonable if all neurons are firing asynchronously, which we expect to be the case due to the heterogeneity imparted by the bias currents.

• 4. The constant synaptic input can be obtained by replacing the synaptic conductances by their temporally averaged (in fast time) values. This is similar to ergodicity and should follow from asynchrony.

Denoting this constant synaptic input by g _syn , we are deriving the mean-field description in three steps.

• 1. We begin by computing the single-cell neuronal firing rate r(g _syn , I) as a function of g _syn and I. Let T = T(g _syn , I) = 1/r be the interspike interval of this neuron. Then solving Eq. (4) analytically gives

  $$T(g_{syn},\!\: I)\!\!=\!\!\left\{\! \begin{array}{ll} \!+\infty & \!\mbox{if } {\Theta}_{eff}(g_{syn},\:I) \!\leq\! 1 \\ \!\tau_\mathit{ref}\!+\!\frac{\tau}{1{\kern-.5pt}+{\kern-.5pt}g_{syn}}\log \left(\! 1\!+\!\frac{1}{{\Theta}_{eff}(g_{syn},\:I)-1} \!\right) & \!\mbox{if } {\Theta}_{eff}(g_{syn},\:I)\!>\!1 \end{array} \right.$$

  (8)

  Here

  $${\Theta}_{eff}(g_{syn},\:I)=\frac{\ensuremath{I}+g_{\mathit{syn}} V_{\mathit{syn}}}{1+g_{\mathit{syn}}}$$

  (9)

  is the effective normalized excitability incorporating both g _syn and I, and

  $${\Theta}_{eff}(g_{syn},\:I)=1$$

  (10)

  defines the effective firing threshold. The corresponding single-cell firing rate surface is shown in Fig. 11(a). It can be used to determine firing-rate profiles of the population for different g _syn -values, for example, at various moments during active and silent phases.

• 2. We now compute the temporally averaged synaptic activation $\hat{q}(g_{syn},\:I)$ for each synapse. We first solve Eq. (6) analytically assuming that synapses are activated for a short period of time relative to the interspike interval. We then require that the synaptic conductance be periodic, consistent with each neuron firing at a constant rate. Finally, we take the temporal average (these steps are carried out in detail in Appendix ). In the end, we obtain the following expression:

  $$\hat{q}(g_{syn},\:I)=r(g_{syn},\: I) \left(c-\frac{d}{e^{\beta_q/r(g_{syn},\: I)}-w}\right)$$

  (11)

  Here c, d and w are positive constants (see Appendix ), and w < 1.

• 3. Using Eq. (11), we calculate the synaptic output as

  $$g_{out}(g_{syn}, \: t)={\overline{g}}_{syn} N^{-1}\sum_{i=1}^N s_i(t)\hat{q}(g_{syn},\:I_i)$$

  (12)

  In the limit of large N, we replace the discrete average over the population by its limit. Assuming that the bias current values are independently drawn from the same probability distribution, we use the Law of Large Numbers to conclude that this limit is the expected value of $s(t, \: \ensuremath{I})\hat{q}(g_{syn},\:I)$ with respect to the distribution of bias currents (s(t, I) is the distribution of slow depression variables). Hence, we obtain the following formula for the function g _out (g _syn , t) that is the normalized network output produced by the steady synaptic input g _syn :

  $$g_{out}(g_{\mathit{syn}}, \: t) = \int_{-\infty}^{\infty} f(\ensuremath{I}) s(t, \: \ensuremath{I}) \hat{q}(g_{syn},\:I) dI$$

  (13)

  Here f(I) is the probability distribution of the bias currents. The quantity g _out (g _syn , t) represents a general way of looking at the network’s input-output relationship for various combinations of cellular and synaptic parameters. However, because of the recurrent connections in the network leading to the synaptic output being fed back as synaptic input, to characterize the steady state(s) of the system, we require the self-consistency condition, i.e., that the input to the network be equal to its output:

  $$g_{\mathit{syn}} = {\overline{g}}_{syn} \int_{-\infty}^{\infty} f(\ensuremath{I}) s(t, \: \ensuremath{I}) \hat{q}(g_{syn},\:I) dI$$

  (14)

To conclude, we have derived the mean-field description (14) by employing a double conductance-averaging process: first, temporal, and then, across the population. It is important to note that the numerical evaluation of the integral in (14) requires special consideration whose details are given in Supplementary Material.

3.3 Bistability in the uniformly depressed heterogeneous network

The time courses in Figs. 2 and 4 show that synaptic depression decreases synaptic efficacy during an episode, until the episode stops, and during the interepisode interval there is a gradual recovery from depression. The abrupt pattern of switching between the high- and low-activity phases suggests that the mechanism for episode generation is based on bistability: the network could be either in a low or high activity pseudosteady state, and synaptic depression switches back and forth between these two states. Additionally, bistability of steady-state network activity for a range of connectivity strengths was the foundation of episodic behavior in the ad hoc model of Tabak et al. (2000) where the introduction of an activity-dependent slow depression variable modulating the connectivity strength allowed for the spontaneous generation of episodes.

Hence, in this section we use the mean-field description developed in the previous section to first look at the steady-state bistability of g _syn . We set all s _i  = 1, so that the network is nondepressed, and study bistability with respect to ${\overline{g}}_{syn}$ in the N-cell model by exploring the properties of g _out (g _syn ) from (14).

Let us denote the steady network input by g _in . Then, the self-consistency condition (14) becomes

$$\frac{g_{in}}{{\overline{g}}_{syn}}=g_{out}(g_{in})$$

(15)

We study (15) graphically. The left- and right-hand sides of (15) are both functions of g _in , and the intersections of their graphs are the self-consistent values of g _syn , i.e., the values of g _syn at steady state. Figure 6(b) shows a graph of g _out (g _in ) and the straight lines representing $g_{in}/{\overline{g}}_{syn}$ from (15) for several ${\overline{g}}_{syn}\text{-values}$. Generally, if ${\overline{g}}_{syn}$ is small [${\overline{g}}_{syn}<0.77$ in Fig. 6(b)], only a low-activity self-consistent value of g _syn exists. On the other hand, if ${\overline{g}}_{syn}$ is large [${\overline{g}}_{syn}>2.2$ in Fig. 6(b)], then only a high-activity self-consistent value exists. For a range of intermediate ${\overline{g}}_{syn}\text{-values}$, it is possible to have three self-consistent g _syn -values, which implies that bistability is probable (the stability of the high- and low-firing steady states was demonstrated by our simulations). The distributions in Figs. 2–4 produce the input-output curves g _out (g _in ) shown in Fig. 6(a). Among these, only those with small γ produce bistability, but it is difficult to visualize. The other distributions are too excitable and do not lead to steady-state bistability.

Fig. 6

Steady-state input-output relationships of the nondepressed or uniformly depressed networks and graphical representation of the self-consistency condition (15) on the recurrently generated synaptic drive. Panel (a) shows nondepressed input-output relationships for the four I-distributions in Figs. 2–4. For small synaptic input g _in , the curves for the same heterogeneity are, approximately, translations of each other along the g _in -axis, consistent with (8) and (9). The intersections of the input-output curves [right-hand side of Eq. (15)] and straight lines that are the graphs of $g_{in}/{\overline{g}}_{syn}$ [left-hand side of (15)] correspond to the steady state(s) of the network. Panel (b) illustrates the self-consistency condition (15): three steady states—low, intermediate, and high (not shown)—exist for ${\overline{g}}_{syn}:\ 0.77<{\overline{g}}_{syn}<2.2$, whereas for ${\overline{g}}_{syn}<0.77$, only the low steady state exists, and for ${\overline{g}}_{syn}>2.2$, only the high steady state exists. The black nondepressed network output curve in panel (b) represents the graph of g _out (g _in ) for I uniformly distributed on the intervals (0; 0.2) and (1.0; 1.2), with γ = 0.2. The sole reason for using this specific distribution was to show the intersections clearly. Among the actual distributions in panel (a), only those with small γ are not very excitable and produce bistability. Changing ${\overline{g}}_{syn}$ as a parameter for a nondepressed network is the same as changing the uniform depression variable s (with ${\overline{g}}_{syn}$ fixed) for the uniformly depressed network

For each value of ${\overline{g}}_{syn}$, we determine the self-consistent values of g _syn as described above. By plotting these values of g _syn vs. ${\overline{g}}_{syn}$, we obtain a bifurcation diagram. If γ is not too large, the bifurcation diagram possesses the S-shape, characteristic of systems with recurrent excitation. Two bifurcation diagrams corresponding to the distributions of bias currents in Fig. 2 are shown in Fig. 7. For each value of g _syn , we also compute and plot the fraction of cells firing within the population (FF). Additionally, simulation-produced counterparts of g _syn and the fraction firing are plotted on the same graph. Generally, both diagrams exhibit good correspondence between the simulation results and analytical predictions, with slight deviations very close to the turning points (where the network is very sensitive to small perturbations in ${\overline{g}}_{syn}$) caused by finite-size effects.

Fig. 7

Steady-state bistability in the nondepressed or uniformly depressed network with large heterogeneity. Plotted are the self-consistent values of synaptic drive g _syn and fraction of population firing (FF). N-cell model simulation results are indicated by thick black and blue lines, mean-field predictions by thin red and orange lines. I-distributions correspond to those in Fig. 2. Small γ in panel (a) leads to bistability, with stable high-activity and low-activity steady states. The middle branches in panel (a) are absent in simulation results due to their unstable character. A large γ in panel (b) makes the network so excitable that bistability is precluded, and only one steady state exists. The trajectory in magenta shows a simulation-produced g _syn (t) vs. $\langle s \rangle(t){\overline{g}}_{syn}$ as the abscissa variable, where 〈s〉(t) is the average of all s _i -values in the network at simulation time moment t [cf. Fig. 2(a)]. The trajectory does not follow the lower branch of the bifurcation diagram in the silent phase (ends near $s(t){\overline{g}}_{syn}=1.2$ (not shown) and not at the right knee). Therefore, the use of the average depression cannot explain episodic behavior in the N-cell model with heterogeneous depression

From this point on, we call the turning points of the bifurcation diagrams the left and right knees, respectively. Let their coordinates be $(\overline{g}_{syn}^L;\: g_{syn}^L)$ and $(\overline{g}_{syn}^R;\: g_{syn}^R)$, respectively. We now look in detail at how bistability is related to the properties of g _out (g _in ) of the nondepressed N-cell model. A necessary condition is for γ to be small enough [as in Fig. 7(a)]. Otherwise, the network is too excitable, and bistability is impossible [Fig. 7(b)]. If γ is sufficiently small, whether the network is bistable depends on ${\overline{g}}_{syn}$. For ${\overline{g}}_{syn}<\overline{g}_{syn}^L$ only the lower steady state L exists, with only a fraction of cells firing. The network excitability is low and hence cannot support anything other than the low-firing steady state. When ${\overline{g}}_{syn}$ becomes equal to $\overline{g}_{syn}^L$ [notice the tangency of the graphs of the left- and right-hand sides of (15) for ${\overline{g}}_{syn}=0.77$ in Fig. 6(b)], a pair of middle (M) and upper (H) steady states is born, presumably via a saddle-node bifurcation (Strogatz 2000), with a larger fraction of the network firing in M than in L, and the whole population firing in H. The left knee corresponds to the minimum level $\overline{g}_{syn}^L$ at which the least excitable cell in the network begins to fire. The sharp corner, often observed at the left knee, is a consequence of the transition from some fraction of the network firing to the whole network firing: no new cells are recruited as g _syn increases through g _syn ^L along the bifurcation diagram (in contrast to the situation for $g_{\mathit{syn}}<g_{syn}^L$), creating the discontinuity in the derivative of g _out (g _in ).

For ${\overline{g}}_{syn}>{\overline{g}}_{syn}^L$, but close enough, the network is sufficiently excitable to support three different firing regimes. As ${\overline{g}}_{syn}$ increases further, M and H move away from each other, until M coalesces with L at ${\overline{g}}_{syn}^R$ in another saddle-node bifurcation [notice the tangency for ${\overline{g}}_{syn}=2.2$ in Fig. 6(b)], after which H only exists. The right knee occurs at the maximum level ${\overline{g}}_{syn}^R$ at which the network can still maintain a low activity steady state. Some fraction of the network is silent at the right knee, which accounts for the absence of any sharp corner there. Above this level of ${\overline{g}}_{syn}$, due to strong recurrent excitation, the entire network is always active, and only the high activity steady state remains.

3.4 Uniformized depression fails to predict key features of the episodic behavior

To understand how bistability could lead to episodic behavior, let us use the ad hoc model of Tabak et al. (2000) as our guide. Consider a single activity-dependent depression variable s, e.g., the average depression across the network. If s decreases (network depresses) when g _syn is large, and increases when g _syn is small (recovery from depression), with proper parameter choices for s we can expect to obtain episodes since varying ${\overline{g}}_{syn}$ for the nondepressed network is the same as varying s for the uniformly depressed network with ${\overline{g}}_{syn}$ fixed.

However, in the N-cell model, the s _i -variables are not uniform. In Fig. 2, though, 〈s〉 seemed to describe the episodic behavior well. To determine whether 〈s〉 can be indeed used to understand episodic behavior in the N-cell model, we run a simulation, compute the average slow depression 〈s〉(t) [cf. Fig. 2(a)] of all s _i -values in the network at each simulation time moment t and plot g _syn (t) vs. $\langle s \rangle(t){\overline{g}}_{syn}$ as the abscissa variable in magenta in Fig. 7(a). Since the simulation and mean-field steady-state bifurcation diagrams in Fig. 7 correspond so well, we might expect the trajectory to follow the lower and upper branches of the bifurcation diagram closely. However, the trajectory does not follow the lower branch of the bifurcation diagram in the silent phase at all, and returns to the upper branch near $s(t){\overline{g}}_{syn}=1.2$ (not shown), very far from the predicted point, the right knee. Therefore, the average slow depression cannot be used to analyze the episodic behavior.

In fact, any single continuous slow depression variable, not only average slow depression, is inadequate for explaining episodic behavior in the N-cell model. Consider the situation in which the bifurcation diagram of the nondepressed network is not bistable [e.g., Fig. 7(b)]. Since the bifurcation diagram has no knees, we conclude that episodic behavior is impossible for the N-cell model with any uniform slow depression s (provided that it is continuous). However, the same network with individual s _i -variables can generate spontaneous episodic activity as was shown in Fig. 2(b). Importantly, this indicates that bistability in the nondepressed N-cell model is not a necessary condition for episodic behavior in the N-cell model with heterogeneous depression. This is in stark contrast to the ad hoc model of Tabak et al. (2000) , in which episodes are impossible if the nondepressed network is not bistable. In fact, we later show that approximately 50% of episodic behavior cases occur when the nondepressed network is not bistable [Fig. 9(a)]. This increase in the robustness of episodic rhythmogenesis is due to the heterogeneity of the slow depression variables that results from the heterogeneity of cellular excitabilities (bias currents).

3.5 Dynamical bistability as the basis of episodic behavior in the N-cell model with heterogeneous depression

In the last section, we showed that for the N-cell model bistability of the nondepressed or uniformly depressed network is not a prerequisite for episodic behavior. Then, what could the episodic rhythmogenesis mechanism in the N-cell model be? We suspect that the heterogeneity of the s _i -variables may render the network (dynamically) bistable, i.e., with the s _i -distribution such as shown in Fig. 2(d), the network could have 2 (stable) steady states. The episodic activity would then result from transitions between the high and low states as before. In order to investigate this, we use the same large-heterogeneity distributions as in Figs. 2 and 7 and the interpolated s _i -distributions obtained from the simulations in Fig. 2. To compute the steady states of the network at each instant of time, we use our mean-field model (14) with those s _i -distributions.

We then plot (Fig. 8) the time courses of these steady states (which, technically, should be called pseudosteady states, since they slowly vary on the depression time scale), together with the time course of g _syn from Fig. 2. As can be seen, at most moments in time we find 3 steady states (low, g _L , intermediate, g _M , and high, g _H ) and they vary with time (g _L and g _H increase and decrease during the silent and active phases, respectively; g _M varies in the opposite way). At the end of the silent phase, g _M meets with g _L . Presumably, they coalesce in a saddle-node bifurcation, and the only remaining steady state is g _H , so g _syn jumps to g _H , and an episode starts. Similarly, at the end of the active phase, g _H and g _M coalesce; g _syn falls back down to g _L , and the cycle repeats. Equally importantly, the simulation-produced time courses of g _syn correspond very well to g _L during the silent phase and g _H during the active phase, consistent with the dynamical bistability description just presented.

Fig. 8

Dynamical bistability as the basis of episodic behavior in the N-cell model with heterogeneous depression. Plotted are the time courses of g _syn (in black) superimposed with pseudosteady state solutions (colored symbols) given by the mean-field description (14) with the s _i -distributions from the simulations in Fig. 2. All parameters are the same as in Figs. 2 and 7. g _L , g _M , and g _H denote the low, intermediate, and high activity pseudosteady states, respectively. The simulation-produced time courses of g _syn correspond very well to g _L during the silent phase and to g _H during the active phase. Independent of whether the nondepressed or uniformly depressed network is bistable, the network with heterogeneous depression exhibits pseudosteady state bistability at each moment in time, with phase transitions presumably occurring via saddle-node bifurcations

Thus, using our general mean-field description (14) together with simulation-produced individual depression time courses, we have uncovered a dynamical bistability in the N-cell model and shown that this bistability underlies the transitions between the high-activity and low-activity states. Note especially that Fig. 8(b) corresponds to the nondepressed network that is not bistable [Fig. 7(b)]. Hence, it is the heterogeneous distribution of slow depression variables (which results from the heterogeneity of I) that imparts this new dynamical bistability onto the N-cell model as the episodic rhythmogenesis mechanism. In the mean-field model of episodic behavior with a uniform slow depression variable or in the ad hoc model, positive feedback from excitatory connections provides bistability, while slow depression provides the phase transition mechanism. In the case of the N-cell model the bistability is provided by both recurrent excitation and heterogeneity of depression and, of course, slow depression provides the switch mechanism, now as orderly recruitment and derecruitment (Figs. 2 and 8). This is a fundamental difference between the ad hoc model and N-cell model.

Finally, we offer an intuitive explanation of how the heterogeneity of the s _i -distribution could induce the dynamical bistability. In the nondepressed case (all s _i  = 1), bistability can be absent when the network is very excitable and its input-output curve lies too high [Fig. 6(a), large γ] for three intersections to be possible [cf. Fig. 6(b)]. A uniform depression would scale down all parts of the input-output curve by the same factor and would thus compress the input-output curve vertically in addition to moving the curve lower. An important consequence of that would be the corresponding decrease in the slope of the middle part of the curve, resulting in only one steady state and precluding bistability and episodic behavior. Let us now consider the effects of having a heterogeneous distribution of s _i -variables. During the silent phase, a heterogeneous s _i -distribution scales down the part of the curve for small g _syn (for which only the intrinsically spiking and the most excitable cells fire) by a greater factor than the part of the curve for large g _syn (all cells fire) because the cells firing in the former case are significantly more depressed than the less excitable cells firing and contributing significantly to the synaptic drive in the latter case. The resulting network input-output curve possesses a stretched intermediate region, which can induce bistability and episodic rhythm (with an appropriate s _i -distribution).

3.6 Dynamic range of episodic behavior: robustness increases with heterogeneity

In this section, we systematically study the dependence of robustness, defined as the dynamic range of episodic behavior, on heterogeneity. We fix the spread of heterogeneity and determine the episodic range as a region in the 〈I〉–${\overline{g}}_{syn}$ plane where 〈I〉 is the average bias current (mean excitability) of the distribution.

The results for the same large (1.0) and reduced (0.4) degrees of heterogeneity as in Figs. 2 and 4 are shown in Fig. 9(a). The most important feature of these results is that the dynamic range of episodic behavior increases with heterogeneity. Furthermore, the episodic range consists of two subregions [separated by the dashed red lines in Fig. 9(a)]. Over the left subregion the nondepressed network is bistable, whereas over the right one it is not. Both regions increase with heterogeneity. The increase of the left subregion with heterogeneity could also be present in the N-cell model with uniform depression. However, the right subregion is a qualitatively new feature that only the N-cell model with heterogeneous depression possesses.

Fig. 9

Dynamic range of episodic behavior: more heterogeneity leads to better robustness. Panel (a) shows the regions over which the network is episodic in the 〈I〉–${\overline{g}}_{syn}$ plane where 〈I〉 is the mean bias current in the distribution. For each point $(\langle I \rangle, \: {\overline{g}}_{syn})$ on the approximate boundaries shown, the true boundary lies within 10% of the shown ${\overline{g}}_{syn}$-range at that value of 〈I〉. Indicated by the arrows, left of the dashed red lines in panel (a) are the regions of bistability of the nondepressed network. Right of the dashed red lines are the regions of increased robustness over which the N-cell model with continuous uniform slow depression cannot be episodic. Larger filled symbols in panels (a) and (b) indicate the same operating points, γ = 0.1 and fraction of time spent in the active phase, either ≈ 0.1 or ≈ 0.5 (see text). Panel (b) shows the percentage of time the simulations spent in the active phase for ${\overline{g}}_{syn}$ determined by the choice of the operating points and 〈I〉 varying

We now comment on the nature of the episodic range boundaries. Above the top boundary, for any given network excitability, the synapses are so strong that the system does not leave the high-activity steady state. This is either due to the dynamical bistability being lost and the only steady state being the high-activity state, or because the synapses cannot depress enough to cause the system to switch to the low-activity steady state. Analogously, below the bottom boundary the network remains in the low-activity steady state either because no other steady states exist or because the synapses cannot recover sufficiently. The vertical left boundary corresponds to no intrinsically spiking cells in the network; hence, episodic behavior is precluded as the network will remain completely silent (possibly, after an initial transient).

There are several additional interesting features that the dynamic ranges in Fig. 9(a) possess. First, only a few intrinsically spiking cells (less than 1%) are sufficient for episodic behavior. Additionally, such small percentages display the most flexibility with respect to episodic behavior in terms of ${\overline{g}}_{syn}$. On the other hand, smaller ${\overline{g}}_{syn}\text{-values}$ display the most flexibility with respect to episodic behavior in terms of 〈I〉.

Finally, we verify that the increase in dynamic range due to heterogeneity includes the regimes that are relevant to developing networks [typically γ is on the order of 0.1: (Wenner and O’Donovan 2001; Yvon et al. 2007)]. We thus compare robustness for the two networks (large and reduced heterogeneity) for identical operating points (the same fraction γ = 0.1 of intrinsically spiking cells and the same fraction of time, either 0.1 or 0.5, that the simulations spend in the active phase). We ask whether the dynamic range is increased by heterogeneity, i.e., if the range of 〈I〉-values is increased, while ${\overline{g}}_{syn}$ is fixed. We have chosen ${\overline{g}}_{syn}$ (two values) so that our operating points have the specified fractions of time active, 0.1 or 0.5; these fractions bracket the range assumed to be physiologically relevant. The results are shown in Fig. 9(b) and further demonstrate that larger heterogeneity increases robustness. The results for γ = 0.5 were similar (not shown).

3.7 Episodic behavior in the homogeneous network with slow depression and noise

Mean-field models, e.g., those of Wilson and Cowan (1972), can be tuned to yield rhythmic network activity. In these models, the “foot” of the network input-output relationship providing nonzero response for low firing rates is said to arise from modest amounts of noise or heterogeneity. One wonders then in what sense those sources of fluctuations (stochastic or quenched disorder) are interchangeable in the generation of episodic behavior. Furthermore, could we obtain robust episodes from a homogeneous N-cell model with noise?

To explore these issues, we consider the homogeneous network, a special case of our N-cell model in which the bias current to all neurons has the same constant mean I plus an independent white current noise (described in more detail in Supplementary Material). This noise prevents spike-to-spike synchrony and leads to a smooth “foot”. The homogeneous model is analyzable and is also close to the ad hoc model of Tabak et al. (2000). If the slow depression recovery and decay rates are chosen appropriately, slow depression should be able to terminate the active phase. Also, as slow depression recovers in the silent phase, low tonic firing rate due to the effects of noise could eventually recruit the network into an active phase.

In the homogeneous network, all neurons are identical except for the noise. To explain the dynamical behavior of this model, we use our mean-field description (14). f(I) in (14) becomes a point mass at I and (14) reduces to

$$g_{\mathit{syn}} = {\overline{g}}_{syn} s \hat{q}(g_{\mathit{syn}}, \: I)$$

(16)

where s is the slow depression variable (mean over the network).

Let r = r(g _syn , I) be the steady-state mean firing rate of a cell in the homogeneous network (Supplementary Material). Since the only differences among the cells are due to noise, which is a fast process with zero mean and small variance, we obtain the following equation for s in terms of r only:

$$\dot{s}=\alpha_s(1-s)-\beta_s\epsilon_s r(g_{\mathit{syn}},\: I) s$$

(17)

This is Eq. (7) in which the second term on the right-hand side has been averaged over one interspike interval. This averaging in (7) is justified provided the cell is not firing too slowly so that its firing can still be considered fast relative to the depression time scale. All the cells always conform to this provision, except for a short period when they just begin to fire. The second condition necessary to justify this averaging is that $\epsilon_s \leq 1/r(g_{syn},\: I)$ so that the depression does not decay at all times. This is automatically satisfied because ε _s  < τ _ref for our choice of parameter values. Equations (16–17) form the complete mean field-type model for the dynamical behavior of the homogeneous model.

We can now find the slow depression nullcline s _∞(r), i.e., the curve on which the right-hand side of Eq. (17) is 0:

$$\begin{array}{*{20}ll}s_\infty(r) & = \frac{\alpha_s}{\alpha_s+\beta_s\epsilon_s r} \\ & = \frac{1}{1+\delta_s r}\end{array}$$

(18)

where δ _s  = β _s ε _s /α _s is an aggregate depression strength parameter. Hence, the slow depression nullcline is a hyperbola, with 1/δ _s a stretching factor along the r-axis.

Now, we can apply phase-plane analysis to the bifurcation diagram of the nondepressed model with ${\overline{g}}_{syn}$ replaced by $s{\overline{g}}_{syn}$ and depression nullcline superimposed [Fig. 10(a)]. The bifurcation diagram here shows the self-consistent values of r (equivalently, the instantaneous mean firing rate $r_{av}=N^{-1}\sum_{i=1}^{N}r(g_{syn},\:I_i)$ of the network), and not g _syn as before, so that both the bifurcation diagram and the depression nullcline are expressed in the same variables. To obtain the bifurcation diagram, we compute and plot the firing rate r at each solution of Eq. (16).

Fig. 10

Phase-plane analysis (panel a) and episodic time courses (panel b) for the homogeneous N-cell model with noise. Shown in panel (a) is the mean network firing rate as a function of ${\overline{g}}_{syn}$ for I = 0.8, I = 0.85, and I = 0.9. Episodic behavior is only possible for I = 0.85: the network is not bistable for I = 0.9, and never recovers from a silent phase for I = 0.8. This indicates that the network’s bias current must be tuned very precisely to achieve episodes. Simulation-produced time courses for I = 0.85 are shown in panel (b). ${\overline{g}}_{syn}=2.0$, δ _s  = 40

The necessary and sufficient condition for episodic behavior is for the depression nullcline to intersect only the middle branch of the bifurcation diagram. If this condition is satisfied, s depresses during an episode, decreasing network excitability and eventually terminating the active phase [Fig. 10(b)]. The presence of noise indeed makes it possible for the network to recover from the silent phase: s recovers the network excitability during the silent phase until the entire population is recruited into an episode. Note the very long silent phases: if positioned to intersect the middle branch only, the depression nullcline must pass very close to the right knee. (The duration of the silent phase could be decreased with a different, more horizontal and higher above the right knee, depression nullcline.)

Most importantly, as Fig. 10(a) shows, the bias current I must be tuned very precisely to achieve episodes. For example, for the given level of noise (σ = 0.1, Table 1), I must not deviate by more than 5% from 0.85. Otherwise, episodes cease to exist. (Strictly speaking, for I ≤ 0.8 (but not for I ≥ 0.9) it is always possible to generate episodic behavior for very large ${\overline{g}}_{syn}\text{-values}$ due to noise. However, that would require non-physiologically large values of ${\overline{g}}_{syn}$ and δ _s as well as extremely careful tuning of δ _s .) We do not expect to see such degree of precision in the developing nervous system, rendering this episodic rhythmogenesis possibility biologically unrealistic. Decreasing the noise level would result in spike-to-spike synchrony. Such synchrony amongst the leaky integrate-and-fire units would disallow episodic behavior. Increasing the noise level could easily preclude bistability by making the single-cell firing rate a strictly concave function of g _syn with the mean firing rate of the network becoming a monotone function of ${\overline{g}}_{syn}$ [cf. Figs. 6 and 7(b)]. In any case, changes to σ do not extend the dynamic range of permissible perturbations to I to any significant degree.

We conclude that episodic behavior is not robust for the homogeneous population of noisy integrate-and-fire neurons. This is a consequence of the presence of only one bias current and is independent of slow depression dynamics, i.e., the depression nullcline. An analytical treatment of the issue of noise vs. heterogeneity is deferred to a future paper; here we explain it more qualitatively. Figs. 2 and 4(a–b) show that in the heterogeneous case the active phase ends at the value of g _syn at which the least excitable cell stops firing. On the other hand, the silent phase ends at a much smaller g _syn -value once the intermediary subpopulation has been recruited. In other words, intrinsically spiking cells are necessary to initiate episodes, whereas less excitable cells are essential for terminating episodes. In the homogeneous N-cell model, the dynamic variables are also distributed (due to noise in this case), but the distributions are much tighter. Furthermore, the order of cells within those distributions changes randomly with time since so does the noise. As a result, while noise effectively prevents spike-to-spike synchrony, all cells in the network are equally excitable, and the episodic behavior of the homogeneous population with noise can be essentially described by that of a single cell with the same value of I. This single cell cannot robustly balance the properties of a range of cellular excitabilities present in the heterogeneous case, which leads to an extremely small dynamic range of I.

4 Discussion

4.1 Overview of results

In this paper, we have introduced a cell-based model (4–7) of spontaneous episodic rhythmogenesis in the developing spinal cord. The model is comprised of an all-to-all coupled excitatory network of N spiking, leaky integrate-and-fire neurons with first-order synaptic kinetics and slow synaptic depression. We have shown with numerical simulations that this network is capable of generating robust episodic behavior for various levels of heterogeneity in the distributions of the bias currents (Figs. 2–4). The recruitment of individual cells into an episode and their derecruitment at its end are both orderly with respect to the bias currents (Fig. 3). Furthermore, we have confirmed (Tsodyks et al. 2000; Wiedemann and Luthi 2003) that a small subpopulation (5–10%) of neurons with intermediate excitability recruited during the silent phase plays a critical role in igniting the active phase through the rapid recruitment of all other less excitable cells and discovered that other critical subpopulations exist as well.

Our treatment includes a systematic derivation from the N-cell model of a general mean-field description (14) that includes macroscopic heterogeneity of cellular thresholds and slow depression variables explicitly. Using the mean-field description together with simulation-produced heterogeneous depression time courses, we have then demonstrated that dynamical bistability is the mechanism responsible for episodic behavior in the full N-cell model. We also showed, by example, that an oversimplified mean-field description (based on uniformized slow depression) can be misleading, i.e., by not containing this crucial dynamical bistability even though the full model has it. Finally, we have demonstrated that increased heterogeneity enlarges the dynamic range of episodic behavior (Fig. 9), i.e., produces better robustness, a necessity for the developing nervous system in which parameters do change with time. We have also explored and analyzed episodic behavior using our mean-field description in a special case of the N-cell model, the homogeneous population with noise. In our model, noise, unlike heterogeneity, does not lead to robust episodic rhythmogenesis with respect to perturbations in the distribution of bias currents.

4.2 Comparison of models for episodic behavior and population spikes

Our N-cell model and results obtained with it share several aspects with those reported by Tsodyks et al. (2000), Loebel and Tsodyks (2002), and Wiedemann and Luthi (2003). Each study demonstrates that spontaneous episodic activity can be produced by heterogeneous integrate-and-fire networks with activity- dependent synaptic depression responsible for the initiation and termination of episodes. Each model is deterministic and in each, recovery from the silent phase involves orderly recruitment proceeding from more to less excitable cells or groups of cells via an important population of intermediate excitability and requires that some fraction of the population be intrinsically spiking, i.e., firing tonically without recurrent synaptic input.

A major difference between our model and the other three is that their episodes, or, “population spikes”, were very brief (each neuron only fired once or twice during a population spike). In those previous studies, synaptic depression develops very fast, on the time scale of one to few spikes. Depression develops more slowly in our model, leading to relatively longer active phases. This feature fits with experimentally observed episodes in the spinal cord (Fig. 1), a neuronal system whose behavior motivates our study. Another difference is that we use full connectivity for computational simplicity whereas the other models used sparse connectivity. Fluctuations due to sparseness might have contributed to the randomness in silent phase duration seen by Wiedemann and Luthi (2003).

We extend the previous works by conducting a systematic study of critical and intermediary subpopulations for several representative choices of network heterogeneity and excitability (Fig. 5).

Furthermore, we demonstrate the importance of the intermediary subpopulation in a new way: the episodic rhythm often stops even if the intermediary subpopulation is replaced by the same percentage of more excitable cells. We also find that the intermediary subpopulation and especially other critical subpopulations can be small, and become smaller as heterogeneity increases, i.e., as the spread in I widens. We believe that this kind of sensitivity is a product of our choice of heterogeneity (uniform I-distribution) and could be significantly alleviated if an appropriate non-uniform, say Gaussian, I-distribution was used.

Additionally, we use our mean-field description to newly identify the mechanism of dynamical bistability underlying episodic behavior. Dynamical bistability, in particular, explains in detail when each phase is initiated and terminated. Most importantly, the emphasis in our work is on the effects of heterogeneity and noise on the robustness of episodic behavior, which were not addressed in previous studies.

4.3 Synaptic depression, cellular adaptation: different features of dynamic variables

An alternative to synaptic depression as an activity-dependent negative feedback mechanism is cellular adaptation. Networks of slowly adapting integrate-and-fire (Latham et al. 2000; van Vreeswijk and Hansel 2001; Giugliano et al. 2004) and Hodgkin-Huxley neurons (Compte et al. 2003) have been shown to generate population bursts. For spontaneous rhythmogenesis, similarly to our model, they utilize either a heterogeneous distribution of bias currents/thresholds so that some fraction of the cells are intrinsically spiking or some injected input (possibly, noise), or both. Here we compare the distributions of the dynamic variables and the resulting episodic behavior for a network with cellular adaptation vs. one with slow synaptic depression.

To develop this comparison, we have run simulations using an N-cell model with I-heterogeneity and slow individual cellular adaptation processes instead of individual synaptic depression processes (unpublished preliminary results). All cells had the same activity-dependent adaptation process. We found that episodic activity, in that case, could be well approximated by simply using the average adaptation over the whole population. Due to cellular adaptation, cells that fire faster adapt more than slower firing cells. Thus, cellular adaptation tends to homogenize the firing rates, and at the beginning of an episode all cells increase their firing rate almost synchronously. Synaptic depression, on the other hand, does not affect the heterogeneity in firing rates, but dynamically decreases the efficacy of synapses as a function of presynaptic firing frequency. The resulting distributions of firing rates and individual synaptic depression variables cannot be described accurately by simply averaging over the whole population. We therefore expect that the effects of heterogeneity would be reduced if cellular adaptation was used instead of synaptic depression. In particular, the increase in the dynamic range to include a region for which the network without negative feedback is not bistable [right of the dashed vertical lines in Fig. 9(a)] is specifically due to the heterogeneity of the slow depression variables. Such an increase should be much less pronounced for cellular adaptation.

However, if the parameters of cellular adaptation were themselves heterogeneously distributed, heterogeneous firing rates would remain. It is not clear whether the effects of heterogeneity described for synaptic depression would still hold in the case of heterogeneous adaptation. However, we note that Wiedemann and Luthi (2003) used heterogeneous adaptation and found no intermediary subpopulation during the recruitment process, in contrast to their findings with synaptic depression. Thus, we speculate that the features observed in the presence of synaptic depression (dynamical bistability, existence of the intermediary subpopulation) will be absent if cellular adaptation is used instead.

4.4 Open issues and generalizations

Our mean-field approach based on fast/slow dissection has worked well to describe the pseudosteady states within the active and silent phases. In a full treatment we would follow the dynamics of the N slow variables during these phases and then describe the rapid transitions between these phases. Temporal averaging yields the approximate N evolution equations [similar to (17)] for the s _i -variables within the phases (Vladimirski 2005) but the orderly dynamics of rapid jumping of cells into or out of high firing rate phases has yet to be systematically worked out. Meanwhile, we have developed some reduced approximate models, say with only three dynamical variables, that are based on our mean-field approach and that capture much of the richness and robustness of episodic behavior in the N-cell model with heterogeneous depression (Vladimirski 2005).

In the developing spinal cord, fast cycling (period of 500 ms or so) occurs within an episode (Fig. 1). Our model, at this stage, does not account for this dynamical structure. The ad hoc model captures this feature with an additional depression process, d in (1)–(2), whose time scale is hundreds of milliseconds, not as slow as that of s. But, in order to achieve cycling in the ad hoc model the time scale of the faster depression could not be much slower than the time scale τ _a [Eq. (1)] of recurrent excitation (Marchetti et al. 2005). Here in our N-cell model, the response time of recurrent excitation (during an episode) should be the fast time scale of V _i and q _i (milliseconds–tens of milliseconds) and this is about 10 times faster than that of cycling. This mismatch of time scales suggests that merely introducing a fast depression (N d-like variables) will be insufficient to account for cycling. The issue of cycle generation in a spiking network model remains open and there is little direct experimental support for candidate mechanisms.

The episodic rhythms that we have modeled are of the relaxation oscillator type, with rapid transitions between the phases. Another class of rhythmic patterns would be slow modulation of firing but without rapid transitions. In this case underlying dynamical bistability may not be necessary to account for the rhythmic activity. Nevertheless, heterogeneity may still be present and important (e.g., for robustness, broad dynamic range). Our mean-field approach would be applicable. The slow modulation could be from, say, sensory input that varies slowly in comparison to spiking and synaptic kinetics, or endogenously generated by opposing slow processes, some for positive feedback and some for negative feedback that could be intrinsic or synaptic (or both) in origin.

Our mean-field framework extends naturally to other neuronal system contexts and can include more biophysical detail when desired. One could incorporate inhibition provided that its time scale is either not much different from the fast time scale of V _i and q _i or is very much slower. The mean-field derivation can be carried out for a wide class of single-neuron models (e.g., Hodgkin-Huxley), synaptic kinetics, slow depression dynamics, bias current distributions, and connectivity patterns. The primary restrictions are: the fast variables for spike generation must remain fast compared to the slow processes; the single-neuron firing rate r(g _syn , I) is computable for every value of g _syn and I and is monotone increasing with respect to each of them for a sufficient range of g _syn -values (if g _syn is large, r(g _syn , I) could even decrease); and the ergodicity-like assumption in the derivation of the mean-field description is valid. Sparse connectivity and synaptic failure, at least in an averaged way, could be included.

Finally, does the particular spiking model affect our robustness findings in an essential manner? The input-output function of the integrate-and-fire model is quite steep near the threshold for tonic firing. Although, during the episodic pattern we see that cells sample this sensitive region only very briefly, during transitions. Therefore, the use of a different cellular input-output relationship could slightly reduce the fragility of rhythmicity of the noisy homogeneous network. However, episodic behavior in the heterogeneous N-cell model is dependent on the distributions of the individual neurons’ firing rates, synaptic conductance, and especially depression variables. For a uniform I-distribution with sufficient heterogeneity, we expect those distributions to be relatively weakly dependent on the single-cell input-output relationship. The main reason is that during the silent phase the cells that do not fire recover from depression independently of their input-output relationship, while the active cells will be much more depressed, similarly to our model with integrate-and-fire neurons. All of the qualitative features of episodic behavior should remain the same and hence we speculate that the robustness will not change significantly. This invariance of robustness might represent some form of self-organization and help to explain why episodic activity is so ubiquitous in the developing nervous system. For the same reason, we also expect that the newly discovered heterogeneity-dependent dynamical bistability for a spiking network contributes in fundamental ways to stability and robustness of episodic behavior for other neuronal systems and models as well.

Appendix

1.1 Details of the mean-field description derivation

To obtain the expression for $\hat{q}(g_{syn},\:I)$ given in (11), we perform the following steps.

• 1. We solve Eq. (6) analytically assuming that synapses are activated for a short period of time relative to the interspike interval, i.e., ε _q  ≤ T(g _syn , I). Using the integrating factor method, we obtain

  $$q_i(t)\!=\!\!\left\{ \begin{array}{ll} {\frac{\alpha_q}{\alpha_q+\beta_q}}\!+\!\left(\! q_i(0)\!-\!{\frac{\alpha_q}{\alpha_q+\beta_q}} \!\right)\! e^{-(\alpha_q+\beta_q)t} & \ 0 \!\leq \!t \!\leq \!\epsilon_q \\ e^{-\beta_q(t-\epsilon_q)}q_i(\epsilon_q) & \ \epsilon_q \!<\! t\! \leq\! T_i \!\equiv \!T(g_{syn},\:I_i) \end{array} \right.$$

  (19)

  Since T _i is the period of neuron i, we must have

  $$\begin{array}{ll} q_i(T_i) & = q_i(0) \\ q_i(0) & = {\frac{\alpha_q}{\alpha_q+\beta_q}}\frac{e^{\beta_q\epsilon_q}-e^{-\alpha_q\epsilon_q}}{e^{\beta_q T_i}-e^{-\alpha_q\epsilon_q}} \end{array}$$

• 2. Find the temporal average of the analytical solution:

  $$\hat{q}(g_{syn},\:I_i)=\!\!\left\{ \begin{array}{ll} \!0, \mbox{ } T_i\!=\!+\infty \mbox{; otherwise:} & \ \nonumber\\ \!1/T_i \int_{0}^{T_i}q_i(t)dt \!=\!1/T_i \!\left( \int_{0}^{\epsilon_q} q_i(t)dt\!+\!\int_{\epsilon_q}^{T_i} \!q_i(t)dt \right) & \ \end{array} \right.$$

  For T _i finite,

  $$\int_{0}^{\epsilon_q} q_i(t)dt = {\frac{\alpha_q}{\alpha_q+\beta_q}}\epsilon_q +\!\left(\! q_i(0)\!-\!{\frac{\alpha_q}{\alpha_q+\beta_q}} \right)\!\frac{1}{\alpha_q+\beta_q}\left(1\!-\!e^{-(\alpha_q+\beta_q)\epsilon_q}\right)$$

  (20)
  $$\begin{array}{*{20}ll}\int_{\epsilon_q}^{T_i} q_i(t)dt &= \left[{\frac{\alpha_q}{\alpha_q+\beta_q}}+\left( q_i(0)-{\frac{\alpha_q}{\alpha_q+\beta_q}} \right)e^{-(\alpha_q+\beta_q)\epsilon_q}\right] \frac{1}{\beta_q}\left(1-e^{-\beta_q(T_i-\epsilon_q)}\right) \\&= \frac{1}{\beta_q}\left({\frac{\alpha_q}{\alpha_q+\beta_q}}\left(1-e^{-(\alpha_q+\beta_q)\epsilon_q}\right) -\left(1-e^{-(\alpha_q+\beta_q)\epsilon_q}\right)q_i(0)\right)\end{array}$$

  (21)

  Hence, combining the similar terms in (20) and (21), we obtain:

  $$\hat{q}_i=r_i\left(c-\frac{d}{e^{\beta_q/r_i}-w}\right)$$

  (22)

  where

  $$\begin{array}{ll} c & = \frac{\alpha_q\epsilon_q}{\alpha_q+\beta_q}+\frac{\alpha_q^2}{(\alpha_q+\beta_q)^2\beta_q}\left(1-e^{-(\alpha_q+\beta_q)\epsilon_q}\right) \\ d & = \frac{\alpha_q^2}{(\alpha_q+\beta_q)^2\beta_q}\left(e^{\beta_q\epsilon_q}-e^{-\alpha_q\epsilon_q}\right)\left(1-e^{-(\alpha_q+\beta_q)\epsilon_q}\right) \\ w & = e^{-\alpha_q\epsilon_q} \end{array}$$

  In general,

  $$\hat{q}(g_{syn},\:I)=r(g_{syn},\: I) \left(c-\frac{d}{e^{\beta_q/r(g_{syn},\: I)}-w}\right)$$

  (23)

  Both (22) and (23) are valid for any firing rate. In particular, for low firing rates, we deduce from (11) that

  $$\hat{q}(g_{syn},\:I) \approx c r(g_{syn},\: I)$$

  (24)

  For high firing rates, by using Taylor’s Formula in powers of $\beta_q/r(g_{syn},\: I)$ and keeping the linear part only, we obtain:

  $$\hat{q}(g_{syn},\:I) \approx \left(c\! - \!\frac{d}{1\!-\!w} \right)\! r(g_{syn},\: I)\! + \frac{b \beta_q}{(1\!-\!c)(1\!-\!c\!+\!\beta_q \tau_{ref})}$$

  (25)

  The β _q τ _ref term in the denominator is the minimum possible value of $\beta_q/r(g_{syn},\: I)$. Thus, the temporally averaged synaptic activation is approximately linear in both cases, with the slope being shallower in the latter. The graph of $\hat{q}(g_{syn},\:I)$ is shown in Fig. 11(b).

Fig. 11

Panel (a): firing rate of the single leaky integrate-and-fire neuron without noise. Transition to repetitive firing occurs when Θ _eff (g _syn , I) = 1, i.e., along the straight line I = 1 + g _syn (1 − V _syn ). If either g _syn or I is fixed at some level, the firing-rate as a function of the other variable possesses the characteristic logarithmic shape. Superimposed are also shown three firing-rate population profiles, corresponding to the three marked steady-state values of g _syn on the lower, middle, and upper branches of the bifurcation diagram in Fig. 7(a). Panel (b): single-cell synaptic activation averaged over one interspike interval as a function of the cell’s firing rate. Note the linear parts for low and high firing rates, consistent with (24) and (25)

We also provide some additional details here. In this work, we use uniform distributions of bias currents on a finite interval (I _min ; I _max ). Then, (14) simplifies to

$$g_{\mathit{syn}}={\overline{g}}_{syn}\frac{1}{I_{max}-I_{min}}\int_{I_{min}}^{I_{max}} s(t, \: \ensuremath{I}) \hat{q}(g_{syn},\:I) dI$$

(26)

For general description purposes (e.g., this is how the bifurcation diagrams in Fig. 7 were generated), we can interpret (14) in terms of ${\overline{g}}_{syn}$ as a function of the self-consistent value of g _syn . This allows us to avoid the numerical solution of (14) altogether, but is not applicable if the self-consistent value of g _syn corresponding to a specific value of ${\overline{g}}_{syn}$ is required. The corresponding expression is

$${\overline{g}}_{syn}=\frac{g_{\mathit{syn}}}{\int_{-\infty}^{\infty}f(\ensuremath{I}) s(t, \: \ensuremath{I}) \hat{q}(g_{syn},\:I) dI}$$

(27)