A simple model of cortical culture growth: burst property dependence on network composition and activity

Abstract

This paper describes large-scale simulations of growth, network formation, and behavior in cultures of dissociated cortical cells. A neuron model that incorporates synaptic facilitation/depression and neurite outgrowth/retraction was used to construct virtual cultures of 10,000 cells whose spiking behavior and evolution were investigated in closed-loop simulations. This approach allows us to perform detailed analysis of the effects of model parameters on burst shape and timing, their changes, and the interrelationship among these behaviors, gross network structure, and model parameters. We examined the effects of two parameters—network composition (fraction of excitatory cells) and neuron excitability (activity level corresponding to neurite outgrowth equilibrium)—on network structure and behavior. Our results suggest that much of the burst shape and timing observed in vitro can be explained by a model that includes only closed-loop neurite outgrowth and dynamic synapses; features such as LTP/LTD, random connectivity, long-distance connections, and detailed neurite topology are not necessary.

1 Background

Cortical cultures grown on multi-electrode arrays (MEAs) have been a compelling method for studying formation of neural circuits for around 30 years (Gross 1979; Pine 1980). They seem to promise to open a window on how the activities and connections of individual neurons contribute to the development and computation of networks. They also allow investigators to examine neural network development (Jimbo et al. 1998; van Pelt et al. 2004a; Wagenaar et al. 2006), to record from and stimulate large numbers of cells (Jimbo et al. 2000), to study the effects of pharmacological agents (Echevarría and Albus 2000), to look at spatiotemporal activity patterns and the relationship between individual cells and the network as a whole (Segev et al. 2004), to explore network responses to stimuli (Jimbo et al. 1998), and to see the relationship between developing connectivity and activity patterns.

In living preparations, network behavior commonly converges to whole-culture bursting behavior (van Pelt et al. 2004a). Besides its theoretical interest (Destexhe and Gaspard 1993; Tsodyks et al. 2000), understanding such activity could have a variety of clinical implications (Wagenaar et al. 2005).

This type of preparation presents a rich set of capabilities for living preparations, for numerical simulation, and for formal analysis. One can examine how the network bursts themselves change as development progresses (van Pelt et al. 2004a; Stegenga et al. 2008; Gritsun et al. 2012), how network behavior depends on experimental parameters such as number of neurons and their density (Wagenaar et al. 2006; Ito et al. 2010) or the fraction of inhibitory cells (Chen and Dzakpasu 2010), or on the correlational relationships between bursts and inter-burst periods (Mok et al. 2012).

In this paper, we present the first large-scale simulations that approach the size and time scale of development in the living preparations and that incorporate a closed-loop growth model in which neural activity can affect neurite outgrowth (which in turn can affect neural activity). Specifically, we incorporate a simple model of activity-dependent neurite outgrowth into a model of cortical cells to simulate the development of networks among 10,000 neurons. We then examine how network activity evolves during development and how its final bursting behavior—both burst shape and inter-burst timing—depends on network parameters.

2 Modeling and simulation

To conduct this investigation, we developed a purpose-built simulator that utilizes graphics processing unit (GPU) hardware to allow large and extremely long-duration simulations to run to completion in reasonable time periods (days or weeks, instead of months or years). This simulation included dynamical neuron and synapse models, an arrangement of 10,000 neurons in a $100 \times 100$ rectangular array, plus a model of activity-dependent neurite outgrowth that modified network connectivity during simulated development. Data collected during simulations included neurite outgrowth information, neuron spiking rates, and network spike counts. We then analyzed these data to identify bursting behavior, if any, to show how burst shape changed during development, and to examine the statistical structure of inter-burst timing.

2.1 Neuron and synapse models

We used a simplified, lumped neuron model that neglected the details of underlying physiological structure, such as ion-channel dynamics and concentration. This neuron model was an integrate-and-fire type and included synaptic, constant bias, and noise currents (Abbott 1999):

$$\begin{aligned} C_m \frac{\mathrm {d} V_m}{\mathrm {d} t} = \frac{1}{R_m}(V_\mathrm {rest} - V_m) + I_\mathrm {syn} + I_\mathrm {inj} + I_\mathrm {noise} \end{aligned}$$

(1)

where $V_\mathrm {rest}$ was both the asymptotic and reset potential, $I_\mathrm {syn}$ was the total synaptic current, $I_\mathrm {inj}$ was a constant depolarizing current, $I_\mathrm {noise}$ was a noise current, and $C_m$ and $V_m$ were the membrane capacitance and resistance, respectively. When $V_m$ exceeded $V_\mathrm {thresh}$, the firing threshold, a spike event was generated and $V_m$ was set to $V_\mathrm {rest}$. This model also incorporated an absolute refractory period, $T_\mathrm {refract}$.

Synapses exhibited dynamics that included activity-dependent facilitation and depression, using the model of Markram et al. (1998) and Tsodyks et al. (1998) with four state variables: three that governed the fraction of synaptic resources in particular states—$x$ (recovered state), $y$ (active state), and $z$ (inactive state)—and one, $u$, that represented synaptic efficiency,

$$\begin{aligned} \frac{\mathrm {d} x}{\mathrm {d} t}&= \frac{z}{\tau _{\mathrm {rec}}} - u x \delta (t-t_{\mathrm {sp}})\end{aligned}$$

(2)

$$\begin{aligned} \frac{\mathrm {d} y}{\mathrm {d} t}&= -\frac{y}{\tau _I} + U x \delta (t-t_{\mathrm {sp}})\end{aligned}$$

(3)

$$\begin{aligned} \frac{\mathrm {d} z}{\mathrm {d} t}&= \frac{y}{\tau _I} - \frac{z}{\tau _{\mathrm {rec}}}\end{aligned}$$

(4)

$$\begin{aligned} \frac{\mathrm {d} u}{\mathrm {d} t}&= -\frac{u}{\tau _{\mathrm {facil}}} + U(1-u) \delta (t-t_{\mathrm {sp}}) \end{aligned}$$

(5)

where $\delta (t-t_\mathrm {sp})$ was the unit impulse at time $t_\mathrm {sp}$, the arriving spike time. The three time constants $\tau _I$, $\tau _\mathrm {rec}$, and $\tau _\mathrm {facil}$ governed inactivation after an arriving spike, recovery from inactivation, and facilitation after a spike, respectively. The synaptic current produced by an arriving spike was $I_\mathrm {syn}=Wy$, where $W$ was the strength of the connection. See tables 8 and 9 in the “Appendix” for parameter values used (Maass et al. 2002).

2.2 Network and growth models

Simulations were conducted by constructing networks with model neurons on a rectangular grid. Connectivity strength (connectivity weight matrix, $W$) was determined dynamically by a model of neurite growth and synapse formation (van Ooyen et al. 1995; van Ooyen and van Pelt 1996). The extent of neurite outgrowth, and thus a cell’s region of connectivity, was modeled as a circle with radius $R_i$ (for cell $i$) that changed at a rate determined by a sigmoidal function of that cell’s firing rate:

$$\begin{aligned} \frac{\mathrm {d} R_i}{\mathrm {d} t}&= \rho G(F_i)\end{aligned}$$

(6)

$$\begin{aligned} G(F_i)&= 1 - \frac{2}{1 + \exp ((\epsilon - F_i )/\beta )} \end{aligned}$$

(7)

where $F_i$ was neuron $i$’s firing rate (normalized to be in the range $[0, 1]$), $\rho$ was an outgrowth rate constant, $\epsilon$ was a constant that sets the “null point” for outgrowth (the normalized firing rate that caused no outgrowth or retraction—one of the network parameters whose effects were explored), and $\beta$ determined the slope of $G(\cdot )$, the function that related firing rate to outgrowth rate. Figure 1 shows $G(F_i)$ for $\epsilon =0.6$ and $\beta =0.1$.

Fig. 1

Dependence of outgrowth/retraction on cell firing rate

This phenomenological model was derived by a number of studies that demonstrated that low levels of electric activity (low firing rate) stimulated neurite outgrowth, and high levels led to retraction (Cohan and Kater 1986; Fields et al. 1990; Schilling et al. 1991; Grumbacher-Reinert and Nicholls 1992). It has been theorized that intracellular Ca$^\mathrm {2+}$ concentration is the underlying mechanism regulating this activity-dependent outgrowth (Kater et al. 1988, 1990). This is conceptually similar to homeostatic synaptic scaling (Turrigiano 2008), formulated as outgrowth or retraction (and thus formation or elimination of synapses), rather than as up or down regulation of receptors at existing synapses.

In the living preparation, development takes place over a period of weeks. To make the simulations described here practical, we used the same value for the outgrowth rate constant, $\rho = 10^{-4}$s$^{-1}$, as van Ooyen et al. (1995). Extensive analysis and simulation were performed to determine the maximum step size that could be used for updating the connectivity with that rate constant, and a value was chosen well below that (100s growth update step size for a 0.1 ms neuron/synapse update step size). This $\rho$ essentially corresponds to a dimensionless time scale; we used a conversion factor of 40 to convert our simulations to 28 days in vitro (DIV).

Overlap between circular extents of neurite outgrowth was used to determine neuron coupling strengths, $W$. Coupling strengths were computed for all pairs of neurons that had overlapping connectivity regions as the area of their circles’ overlap; this caused connections to be created and destroyed dynamically as the simulations progressed. In effect, we assumed that the circle of connectivity for a single neuron was uniformly filled with neurites and that overlapping areas produced synapses with uniform density. Thus, the connection strength between two neurons can be considered to be the net effect of a number of synapses proportional to the areas of overlap. Over the time course of these simulations, each neuron developed connections to, on average, 33 to 127 other cells.

The bulk of the neurons in the network were excitatory; a fraction were chosen to be inhibitory—this fraction was one of the network parameters whose effects were explored. Similarly, most neurons were not spontaneously/endogenously active, but a few (10 %) had their firing threshold, $V_\mathrm {thresh}$, lowered from 15 mV to a value chosen from a uniform distribution in the range 13.565 mV $\le V_\mathrm {thresh} \le 13.655$ mV to produce spontaneous firing at a rate of between 0.02 and 6 spikes/s.

We used the set of standardized, $10 \times 10$ layouts of endogenously active and inhibitory neurons presented in Fig. 2, repeating them in a $10 \times 10$ pattern to fill the $100 \times 100$ arrangements. These layouts were chosen to maximize the distance between nearest inhibitory cells and endogenously active cells, and to allow for comparison of different simulations by providing them with the same layouts if they had the same fraction of excitatory cells.

Fig. 2

Layouts used for different numbers of endogenously active (EA, black dots) and inhibitory (INH, striped dots) neurons. Non-endogenously active, excitatory neurons shown as gray dots. a 10 % EA, 2% INH, b 10 % EA, 10 % INH, c 10 % EA, 20 % INH

2.3 Neural culture simulator

Figure 3 presents the simulations’ algorithmic structure. Each simulation proceeded as a sequence of 100 s activity epochs (inner loop). Within each epoch, connectivity was kept constant and neuron and synapse states were updated with a 0.1 ms time step according to Eqs. (1)–(2). Neuron spike histories (neuron number and firing times) were accumulated for the entire epoch. At the end of each epoch, the average firing rate of each neuron was calculated from its spike history and that average was used to adjust its neurite outgrowth according to Eqs. (6) and (7). Areas of overlap were then computed and synaptic weights were adjusted; this caused synapses to be created when corresponding weights first became non-zero.

Fig. 3

Simulator algorithm structure. A 0.1 ms time step included neuron and synapse updates and was repeated 1,000,000 times to produce a 100 s activity epoch. Activity epochs, separated by growth updates, were repeated to produce 28 DIV simulations

Simulations consisted of networks of 10,000 neurons in $100 \times 100$ arrangements simulated for 600 epochs (corresponding to 28 DIV). This duration was chosen by running test simulations and examining the connectivity radii of all neurons as a function of time to detect stationarity. Because of the smooth, slow variation of radii along time, their derivative was used as a test for stationarity. The derivatives converged to small values for some sets of simulation parameters, while for others there was no indication of such convergence. The criterion for acceptable stationarity was the observation of radii varying less than 15 % during 100 epochs (Stiber et al. 2007). The simulation length of 600 epochs was chosen as a reasonable compromise that included a usable segment of stationary behavior for those simulations that converged.

Preliminary simulations indicated that, while the equilibrium values of neuron connectivity radii vary based on parameters and neuron type and location, generally the ratio between the number of neurons and the number of connections stabilized at around 1:46 (keeping in mind that “connection” here corresponds to multiple synapses in a living preparation). Therefore, the states of 10,000 neurons and about 460,000 “synapses” were required to be updated every time step using in this case an exponential Euler method. Consequently, a 600 epoch simulation of a $100 \times 100$ arrangement represents $6 \times 10^8$ time steps, or approximately $2.8 \times 10^{14}$ state updates.

The simulator was implemented as a single-threaded program and as a parallel, multi-threaded program using OpenMP. Benchmarks indicated that the single-threaded implementation of a $100 \times 100$ network would take at least 2,000 h (83 days). This work also suggested that parallelization using OpenMP could not feasibly lead to a large enough speedup to make such simulations practical, and thus, we turned to a GPU implementation using compute unified device architecture (CUDA).

CUDA is a parallel computing architecture developed by NVIDIA, which enables GPU programming through a C-like programming language. In the CUDA model, code that is to execute on the GPU is written as a “kernel” and called from host (CPU) code as a function. When the kernel function is called, the CUDA runtime generates a large number of threads to exploit data parallelism using the parallel GPU hardware. The final simulator version runs almost entirely on the GPU, producing an approximately 23 times speedup compared with the single-threaded version (Kawasaki and Stiber 2012). Simulations were run on NVIDIA Tesla C1060 and M2090 devices.

We collected and recorded:

• radius history Radius of each neuron for each activity epoch.

• rate history Average firing rate of each neuron for each activity epoch.

• spike history Number of spikes produced by the entire network in each 10 ms time bin for the entire simulation. We did not record each individual neuron’s spike history in 10 ms bins because of memory constraint—it would require about 480 GB to store 10,000 neurons’ spike histories for a 600 epoch simulation using 10 ms bins—and because it was not necessary for the current investigation.

3 Data analysis

Three categories of data analyses were done. We first plotted the raw data and simple statistics to provide an overview of developmental changes during the entire course of simulated development. We then used a detection algorithm to capture burst times (if any) and network activity during bursts. This information was then used to describe network intra- and inter-burst activity.

We chose two parameters (fraction of excitatory neurons and the null point for outgrowth, $\epsilon$) and observed network behavior as these were varied. We ran 600 epoch (28 DIV) simulations with the set of parameters (fraction excitatory cells, target rate) = (0.8, 0.1), (0.8, 1.0), (0.8, 1.9), (0.9, 0.1), (0.9, 1.0), (0.9, 1.9), (0.98, 0.1), (0.98, 1.0), and (0.98, 1.9).

3.1 Basic analysis

As metrics of the neural network behaviors, we calculated and plotted measures of neuron activity and neurite outgrowth versus time. In particular, we considered radius history and rate history as described previously. These were plotted against time for all neurons in what we called basic plots.

We also computed average per neuron firing rate (APNFR) from the network spike history: spike count of the entire network per second normalized per neuron and expressed as Hz/neuron, derived by the formula $(\text {network spike history})/(0.01 \times 10,000)$. The entire history of each simulation was summarized by a simple APNFR histogram, plotted on log-log axes because of the wide range of rates and counts (4–5 orders of magnitude in both cases). APNFR was also plotted as a function of time along each simulation, in which the raw APNFR values were averaged to produce 1,000s time resolution.

Besides plotting radius histories of all neurons together versus time, we also plotted the mean and standard deviation of all excitatory (excluding edge neurons) and inhibitory neurons’ radii versus time using 1,000s windows.

3.2 Burst shape

The extremely long simulation lengths prevented examination of bursting via the basic plots of neuron firing rates simply due to the enormous resolution that would have been required. Instead, we used APNFR to identify bursting. We established a firing rate threshold to separate bursting from background firing using the APNFR statistics summarized by the histograms shown in Fig. 5. Note that this figure displays two types of quantization: along the $X$-axis, these are the bin widths (wide to the left; becoming invisibly narrow to the right) and along the $Y$-axis, an increment representing a difference of 1 in the bin counts (noticeable only for small counts).

More specifically, we compared the distributions of APNFR for segments of simulations known to not contain bursting to those for segments known to contain bursting. From this, we determined that the mean inter-burst APNFR was 0.21 (spikes/second/neuron); intra-burst APNFR averaged around 1. We thereby established a firing rate threshold to detect burst initiation and cessation to minimize the probability of false positives while also minimizing burst timing errors. For simulations that produced bursting, during non-bursting periods APNFR was also 0.21 with a standard deviation of 0.04; we chose 0.5 spikes/second/neuron as a threshold for burst detection—more than 7 standard deviations above the mean (Kawasaki 2012). This value excluded almost all background activity but still allowed detection of bursts in the early simulation period, when their heights were usually lower. Figure 4 shows example bursts from early (A) and late (B) simulation times. This choice of burst detection—statistical separation of burst behavior from background firing—is one of three basic approaches for burst detection in the literature [for example, van Pelt et al. (2004b)]. A second identifies bursts via peak firing rate and then uses half that rate as the definition of burst start and end [for example, Gritsun et al. (2010)]. Other investigators [such as Chen and Dzakpasu (2010)] use hand-tuned thresholds for burst detection. Variation in these methods can be assumed to produce some variation in burst shape results.

Fig. 4

Example burst patterns. Plots of APNFR versus time from beginning of burst for bursts at around 5 DIV (a) and 24 DIV (b) with parameters (target rate, fraction excitatory cells)$\,=\,$(1.0, 0.98)

Once we identified bursts, we could characterize them. Bursts could be characterized by intra-burst profile, or shape, and inter-burst profile, generally statistics derived from inter-burst intervals, but also including measures of serial dependency of each interval on previous ones.

The shape of a burst was described using width, peak height, peak position, and spike count. The width was the duration of a burst, measured from a positive-going crossing of the detection threshold to the subsequent negative-going crossing. Peak height was the maximum APNFR within a burst and peak position was time from burst start to that peak. Spike count per burst was the total number of network spikes in the burst normalized per neuron.

To explore burst shape evolution during each simulation, we divided 600 epoch simulations into windows of 10 epochs, calculated the mean values of these burst profile statistics, and plotted them versus simulation time.

3.3 Inter-burst behavior

Inter-burst profiling was based on burst peak times, $t_{b_i}$, $i = 0, 1, 2, \ldots$), reducing the sequence of bursts to a point process. We applied a variety of basic measures of point process evolution, variation, and correlational structure:

• burst “ratime” (rate vs. time) plots We plotted the number of bursts per second along simulation time.

• basic inter-burst interval (IBI) statistics The differences between successive times in the point process is the inter-burst interval ($\mathrm {IBI}_i = t_{b_{i+1}} - t_{b_i}$), which is the reciprocal of the “instantaneous” burst rate. We computed the coefficient of variation (CV, $\sigma /\mu$) of the IBIs and created an IBI histogram (IBIH) for the last 100 epochs of each simulation using a 1s bin size.

• IBIH characterization We characterized the shape of the IBIH by fitting a generalized extreme value (GEV) distribution, in part to facilitate comparison with the literature (Gritsun et al. 2011). The GEV distribution is a family of continuous probability distributions developed with extreme value theory to combine type I (Gumbel), type II (Frechet) and type III (Weibull) extreme value distributions. Distributions whose tails decrease exponentially, such as normal distributions, lead to type I. Distributions whose tails decrease as a polynomial, such as Student’s t, lead to type II. Distributions whose tails are finite, such as the beta, lead to type III. The GEV distribution is characterized by three parameters: shape ($\xi$), scale ($\sigma$), and location ($\mu$). The $\xi$ parameter determines the form of the GEV distribution: $\xi =0$, type I; $\xi >0$, type II; and $\xi <0$, type III. We used the MATLAB mle() function to get maximum likelihood estimates (MLEs) for the parameters of GEV distributions from the IBIs (The MathWorks Inc.). Then, we used the MATLAB gevpdf() function (generalized extreme value probability density function) to plot the probability density function of the GEV distribution with parameters returned by the mle() function on top of the IBIHs for the last 100 epochs of each simulation. We also separately plotted the evolution of the GEV distribution $\mu$ and $\sigma$ along simulation time using 100 epoch bins.

• IBI temporal patterns Serial correlations in bursting (for example, periodicities) were evaluated first by computing the power spectrum for the sequence of IBIs (Brillinger 1975; Brillinger et al. 1976; Rigas 1992). In application to point process intervals, frequencies are relative to burst number, $i$ (count, also termed “order”), rather than time (though of course events happen along time, so their count relates to the passage of time). Peaks in the power spectra corresponded to periodicities in the IBIs. The highest possible frequency observable would be 0.5 cycles/order, corresponding to a periodicity of two (short-long-short-long-$\cdots$). A second measure of temporal patterns involved the creation of return maps of IBIs, which were reduced dimensionality representations of the underlying dynamical attractors (Rapp et al. 1985; Bergé et al. 1986). In these maps, $\mathrm {IBI}_{i+1}$ was plotted against $\mathrm {IBI}_i$ to create a first order interval return map. More generally, the $j^\mathrm {th}$ order interval return map would be the plot of interval $i+j$ versus interval $i$. A summary of the use of return maps in diagnosing different deterministic or stochastic behaviors is beyond the scope of this section, but suffice it to say that purely stochastic behaviors generally produce non-systematic scatterings of points while deterministic behaviors—be they periodic, quasi-periodic, or chaotic—will generally produce distinctive patterns. Combinations of stochastic and deterministic elements will often produce patterns that are characteristic of the deterministic element, with additional stochastic scatter.

4 Results

Results presented here are from a set of 9 simulations that explore the parameter space described by fraction excitatory cells $= (0.8, 0.9, 0.98)$ and target rate $\epsilon = (0.1, 1.0, 1.9)$.

Figure 5 presents average per neuron firing rate (APNFR) histograms for all simulations. Each histogram displays a large peak at a low rate around 0.2 Hz, corresponding to the background (non-bursting or inter-burst) neural spiking behavior, and a secondary peak at a much higher rate (between 10 and 100 Hz), corresponding to intra-burst spiking. The burst detection threshold, 0.5 Hz, can be seen to be, in all cases, well above the bulk of the background firing distribution.

Fig. 5

APNFR histograms for the entire simulation period for given $\epsilon$ (indicated at left) and fraction excitatory cells (indicated above), log-log axes. APNFR bin size 0.01 Hz

4.1 Basic plots

Figure 6 shows plots of radius and rate histories along simulation time for each of the nine simulations. For the sake of conciseness, only plots for excitatory, non-spontaneously active, interior, neurons are presented. These plots include a separate curve for each of the roughly 9,300–9,500 such neurons; any spread illustrates the range of behaviors, though their distribution is obscured by the limited plotting resolution. In most cases, final firing rates (lower plots) converged onto the target rate, $\epsilon$, except for spontaneously active neurons (not shown) with firing rates above the target rate. Those neurons did not slow down, though slower firing spontaneously active neurons did speed up. As a result, while most neurons’ firing rates converged toward the target, spontaneously active cells’ rates invariably filled the range from the target rate up to the maximum the individual thresholds supported (in these simulations, around 6 Hz).

Fig. 6

Basic plots of connectivity radius (top, each part) and firing rate (bottom, each part) for the entire simulation period for given $\epsilon$ (indicated at left) and fraction excitatory cells (indicated above)

Observations of APNFR were consistent with those of the individual neurons’ firing rates. Rates stayed near their spontaneous value (0.21 Hz per neuron) until neurons grew to connect to each other in a wide enough neighborhood, sometime around 5 DIV. Note that this APNFR does not imply that all cells were firing. To be precise, only the spontaneously active cells were firing initially; since these represented 10 % of the entire network, this APNFR corresponded to their average rate of 2.1 Hz.

Once neurons were sufficiently interconnected, non-spontaneously firing neurons started firing and APNFR increased dramatically to around the target rate parameter used for neurite outgrowth. This was true even for those simulations that did not appear to have stationary radii near the end of the simulation (as described in Fig. 7. In those cases (fraction of excitatory cells = 0.8), APNFR declined slowly during the course of the simulation even as connection radii increased.

Fig. 7

Evolution of average radii, and standard deviation of radii (“error” bars), for all simulations (solid line for excitatory, non-spontaneously active, interior neurons, and dashed line for inhibitory neurons). $\epsilon$ indicated at left and fraction excitatory cells indicated above

Connectivity radii for edge and corner neurons were larger than those shown, probably due to them having fewer neighbors and thus less input stimulation. Inhibitory neurons generally had moderate radii, except for simulations with the most inhibitory cells (fraction of excitatory cells = 0.8), where they had larger radii. Spontaneously active neurons had a wide range of different connectivities, likely due to the variability in their firing thresholds.

Large, broad “bursts,” with enormous APNFR corresponding to all of the cells in the network firing at high rate for hundreds of seconds with multi-epoch quiescent periods between, were exhibited by the simulation with the fewest inhibitory cells and lowest target rate (fraction of excitatory cells = 0.98, $\epsilon = 0.1$). These “bursts” extended over many simulation epochs and coincided with long-duration declines in interconnection radii; the “inter-burst” intervals, also many epochs in duration, coincided with steady increases in interconnection radii. This pretty clearly represented a hysteretic switching of the network between two firing behaviors, rather than actual bursting, and thus, this simulation was excluded from most subsequent analyses.

Note that, excepting that indicated above, bursting behavior is not apparent in the firing rate plots due to their low resolution compared with burst duration. However, APNFR analysis invariably detected burst initiation at around 5 DIV. This appears to correspond to radii of around 1.5, which would be around the value at which neurons a distance of 2 apart diagonally first form synapses, suggesting that the minimum neighborhood size for initiation of bursting was greater than four-connected.

4.2 Connectivity and firing rate evolution

Figure 7 summarizes the average and standard deviation of connectivity radii for non-endogenously active, non-edge excitatory and inhibitory neurons as a function of time. Each simulation was broken into non-overlapping 100-epoch segments, with radii for each neuron of the particular type (non-endogenously active, non-edge excitatory or non-endogenously active, non-edge inhibitory) averaged together to yield a single value for that type. In all cases, neurons’ radii grew steadily and linearly until some time around 5 DIV, when interactions among cells generally increased cell activity and therefore decreased neurite outgrowth rate.

Except for those simulations with the highest fraction of inhibitory neurons, excitatory cells had wider connectivities from after around 5 DIV onward. For at least four of the simulations (fraction excitatory cells = {0.9, 0.98}, $\epsilon = \{1.0, 1.9\}$), it can be seen clearly by eye that radii are stationary by the end of the simulation. These were all characterized by an overshoot in connectivity radii of inhibitory cells between around 5 and 10 DIV and very little variation in excitatory or inhibitory cell radii across the network.

For simulations with the greatest fraction of inhibitory cells, not only was the average inhibitory cell radius greater than that of the average excitatory cell, but both sets of radii were still increasing at the end of the simulation. Inhibitory cells in those simulations also had much greater variation in their radii and higher firing rates. This suggested a positive feedback effect, in which greater connectivity drove greater inhibition, preventing cells from reaching their target firing rate and thus spurring them to further neurite outgrowth.

4.3 Burst shape

The basic idea that burst shape changed as simulations progressed was introduced in Fig. 4. Figures 8, 9, 10, 11 show this shape evolution in detail for all simulations. Figure 8 presents an overview of this evolution, plotting mean burst shapes in 10 epoch time windows. As networks grew, bursts were observed starting at around 5 DIV for all simulations. Networks with higher target rate tended to start bursting earlier, likely from the slightly faster growth rate that this produced. Early bursts were typically wider and lower, gradually evolving into narrower and more intense ones with longer rising phase until (and if) stationarity was reached.

Fig. 8

Evolution of burst shape for all simulations. $\epsilon$ indicated at left and fraction excitatory cells indicated above

Fig. 9

Evolution of mean burst width (solid curve) and mean peak position (dashed) for all simulations. $\epsilon$ indicated at left and fraction excitatory cells indicated above

Fig. 10

Evolution of mean spike counts per burst for all simulations. $\epsilon$ indicated at left and fraction excitatory cells indicated above

Fig. 11

Evolution of burst rates (bursts/s) for all simulations. $\epsilon$ indicated at left and fraction excitatory cells indicated above

Generally speaking, burst peak APNFR ranged from 40 Hz/neuron to 180 Hz/neuron. For the four simulations with target rates of 1.0 and 1.9 previously determined to appear stationary by simulation’s end, burst heights were 70–80 Hz/neuron (for fraction of excitatory neurons = 0.98) and 95–100 Hz/neuron (for fraction of excitatory neurons = 0.90). This is consistent with firing rates observed in the literature [for example, Gritsun et al. (2012)], under the assumption that each electrode in a multi-electrode array records spiking from around one or two neurons.

Simulations (0.80, 1.0) and (0.80, 1.9) show bursts that appear to be in the process of disappearing toward the end. This is consistent with the plots of connectivity radii in Fig. 7, which show increasing radii and declining firing rate.

The large, long-duration high-firing-rate behavior of (0.98, 0.1) shows itself clearly (after passing through a developmental phase of intense, short-duration bursting). Note that now that we are plotting rates only within bursts; the network activity in that case is revealed to be similar in intensity to the “true” bursts shown in the other simulations. We tentatively conclude that, while the other simulations produced behaviors in which burst initiation and termination were a stationary result of short-time-scale dynamics, in this particular simulation changing connectivity was the variable that switched neuron behavior between low and high firing rates.

As shown in Fig. 9 and Table 1, true bursts ranged in duration from tens to hundreds of milliseconds in duration. Thus, interestingly, a rough doubling of target rate did not result in much difference in burst shape. Width data for simulations (0.80, 1.0) and (0.80, 1.9) again suggest that their burst activity was in the process of being extinguished as interconnection radii increased.

Table 1 Mean burst height (Hz per neuron) and burst width (s) during final 10 simulation epochs (equivalent to around 28 DIV)

The width data in Fig. 9 clearly shows the initial, very broad (250–500 ms) bursts that fairly quickly sharpened up to much shorter duration (less than or around 200 ms). Burst peaks invariably occurred at least half-way through the burst. During early development, peaks occurred late in the burst (65–70 % of the way through the burst), indicating that burst built up slowly. At later phases of development, peaks were much closer to midway through the bursts, consistent with faster build-up.

How bursts with durations significantly less than a second contributed to APNFR is partially explained by Fig. 10, which shows the average number of spikes that a neuron contributed to each burst (more precisely, the number of spikes per burst divided by the number of neurons in the network). Over broad ranges, and despite changes in burst width, spikes per burst was generally close to constant, with early wide, low bursts being composed of a similar number of spikes as late, brief, intense bursts. Again, for the four stationary simulations, differences in target rate produced very little difference in this measure of burst shape.

As in the other plots, the simulation at (0.80, 1.0) is especially notable due to the apparent rapid extinction of bursting near the end of the simulation.

4.4 Inter-burst timing

Figure 11 shows evolution of burst rate (bursts/s) with varying simulation parameters. Networks with lower target rates generated fewer bursts per second, and networks with higher target rates generated higher burst rates. For the four simulations that clearly resulted in growth stationarity—(0.90, 1.0), (0.98, 1.0), (0.90, 1.9), and (0.98, 1.9)—the burst rate scaled with the target rate. Thus, the networks’ behaviors responded to different target rates by altering the rate of burst production, rather than changing the burst shape. For (fraction excitatory cells, target rate) = (0.90, 1.9), one can see most obviously an overshoot of bursting at the early period of development.

In the case of (0.80, 1.0) and (0.80, 1.9), burst counts increased dramatically near the simulation end. Note that burst height (Fig. 8), burst width (Fig. 9), and spikes per burst (Fig. 10) all varied at the same time; this appears to have compensated for the increased burst count to keep APNFR stable—the result was more frequent, “smaller” bursts with fewer spikes per burst.

Figure 12 shows distributions of inter-burst intervals (IBIs). The shapes of most IBI distributions were skewed (more short IBIs than long ones). Qualitatively, their appearance is consistent with that of Poisson or renewal processes with dead time (and consistent with living preparations, if one considers the smaller bin size possible here due to the order of magnitude greater number of bursts recorded due to our simulation methods). Networks with lower target rates generated longer IBIs, as expected. Interestingly, variability was greater for lower target rates, as can be seen in the coefficients of variation in Table 2. Networks with higher target rates had more frequent, more regularly spaced bursts. For the four stationary simulations, almost doubling the target rate resulted in a near halving of the CV, suggesting that the mechanism for burst timing variability was largely independent of target rate in those cases.

Fig. 12

Distribution histograms of IBIs (IBIHs) between 23 and 28 DIV, with overlaid fitted GEV functions. $\epsilon$ indicated at left and fraction excitatory cells indicated above

Table 2 Coefficient of variation (CV) of IBI distribution during 23–28 DIV

The effect of increasing inhibition on IBI distribution was more complex. Increasing inhibition led to a decrease in the distribution’s mode and, often but not always, its mean. This was consistent with Fig. 11, though some of this change was due to burst extinction for fraction of excitatory cells  =  0.8. Increasing inhibition also generally led to an increase in CV—inhibition made bursting more irregular.

Figure 13 shows how the location and scale parameters of the generalized extreme value (GEV) distributions fitted to the IBIHs evolved as the simulations progressed; Table 3 gives their values (and that of shape) for the final 100 epoch window as illustrated in Fig. 12. As the 95 % CIs show, all but those with a target rate of 0.1 were fit well by the GEV. The figure shows a rapid change in parameters due to extinction of bursting behavior for the (0.80, 1.0) simulation and the apparent stationarity of the simulations with parameters (0.90, 1.0), (0.98, 1.0), (0.90, 1.9), and (0.98, 1.9). Based on the parameter values in the table, IBIHs could be categorized into the three distribution types:

• Type I (exponential tails) The (0.98, 1.9) simulation had a shape parameter with confidence interval spanning zero and thus was arguably type I.

• Type II (polynomial tails) The simulations with target rates of 1.0 and 0.1 fell into this category, with positive shape parameters.

• Type III (short tails) The (0.80, 1.9) and (0.90, 1.9) simulations had negative shape parameters and so fell into this category. As shown in Table 2, the CVs for these IBIHs were increasing (the distributions getting wider), but their tails were not getting longer; hence, they were categorized differently than (0.98, 1.9).

Power spectra of inter-burst intervals were also calculated to address the issue of whether these IBI sequences were produced by periodic processes. Except for the (0.1, 0.98) case (in which the behavior switching of the network—essentially, three giant “bursts”—was detected), all spectra were flat, indicating no serial correlations and no evidence for periodic structure in the IBI sequences. Note that this does not rule out a deterministic, chaotic, process; only periodic processes.

Fig. 13

Evolution of GEV distribution parameters for inter-burst intervals (solid line for location ($\mu$)—corresponding to distribution mean—and dashed line for scale ($\sigma$)—corresponding to distribution variance; 95 % CIs shown as error bars). 100 simulation epoch, non-overlapping time windows used. $\epsilon$ indicated at left and fraction excitatory cells indicated above

Table 3 GEV distribution parameters (including 95 % CIs for the parameters) between 23 and 28 DIV for each simulation

The burst interval return maps of Fig. 14 also support the assertion that the burst timing was the result of a stochastic process for most simulations. For the four clearly stationary bursting simulations, point scatter is consistent with the interval histograms of Fig. 12 with only a non-deterministic scatter, dead time decreasing with increased inhibition, and no apparent preferred sequence of intervals.

Fig. 14

First order return maps of inter-burst intervals for 23–28 DIV. $\epsilon$ indicated at left and fraction excitatory cells indicated above

The return map for (0.80, 1.9) suggest that there may have been an element of periodicity involved, with long intervals between 1 and 2 s followed by short intervals of less than 0.5 s, and vice versa. This is revealed by the fairly distinct clusters near the X and Y axes. More detailed examination revealed that these clusters extended throughout the 23–28 DIV interval plotted. We hypothesize an explanation, supported by a lack of a peak in this simulation’s IBI power spectrum, that, for the most part, IBIs were stochastic and between 1–3 s. However, occasionally, the system’s state was perturbed into the neighborhood of a period-2 attractor with an IBI between 1–2 s. A sequence of IBIs alternating between 1–2 s and 0–0.5 s was then produced, until an interval greater than around 2 s was produced (note the cluster near the Y axis extended up to around 3 s), at which point the system’s state escaped that attractor’s neighborhood and reentered the apparently stochastic region of its state space.

5 Discussion

We have presented results of activity-dependent growth and neural activity integrated together in closed-loop simulations of dissociated cortical tissue cultures for their full developmental time span and for realistic network size. This was used to explore a simple, activity-dependent growth model that doesn’t involve neurite guidance or LTP, to see how much of network behavior can be explained. This is complementary to recent work by Gritsun et al. (2012) that examined a pair of open-loop simulations: one generating neurite outgrowth via chemotaxic guidance and one using the resultant connectivity to examine network spiking behavior. In the current simulation, however, the network changes itself through its own activity. Moreover, this simulation allows us to investigate just how simple a model of network formation is sufficient to reproduce observed bursting behavior. In general, we show that complicated development/connectivity models (such as those incorporating LPT/LTD, random connectivity, long-distance connections, or detailed neurite topology) are not necessary to explain network bursting, to replicate many bursting characteristics, nor to replicate their evolution. A closed-loop model that couples neurite outgrowth with neuron activity and includes simple long-time-scale dynamics (here, synaptic facilitation and depressions) is sufficient.

Even with the relatively small number of simulations completed here, some conclusions can be drawn regarding the effects of two simulation parameters, fraction of excitatory cells and target firing rate, on overall network behavior. These two parameters relate to two different aspects of the simulation: network structure (in terms of the balance of excitation and inhibition) and neurite outgrowth (and thus extent of interconnectivity). In this discussion, we describe the influences of these parameters on network behavior (burst shape and timing), network connectivity patterns, the potential influence of each on the other, and compare these results with previous investigations of activity-dependent neurite outgrowth and bursting activity in living cortical tissue cultures and short-duration simulations.

By comparison with previous results for small networks, the work here clearly shows the importance of network scale for behavior (Stiber et al. 2007). That earlier work was largely identical to the current report in methods, with the exception of using smaller, 100-neuron networks. With such small networks, only global bursting driven by a hysteresis relationship between the slow variable (growth rate) and the average electrical activity in the network (Babloyantz and Destexhe 1991; Destexhe and Gaspard 1993), such as that exhibited here by (0.98, 0.1), occurred. These kinds of oscillations in connectivity were similar to those observed in smaller, 16-cell simulations (van Ooyen and van Pelt 1996).

It is important to note that both connectivity patterns and network spiking behavior are emergent properties of network changes as directed by simulation parameters. Individual cell firing rates affect connectivity patterns, mediated by target rate, which in turn affects firing rates and, eventually as development progresses, burst shape and timing. The bursts themselves affect neuron firing rates and thus connectivity. Similarly, the distribution of inhibition affects firing rates, connectivity, and bursting in a feedback loop. Moreover, the model employed is fairly parsimonious and thus reduces the space of mechanisms likely to be essential for generating the bursting behaviors seen in the living preparations.

5.1 Burst shape

While we can compare evolution of burst shape in a qualitative manner, it is difficult to make precise quantitative comparisons, due to the fact that experiments must sample network activity via electrode arrays. Other large-scale simulation work also chooses to report sampled data, rather than whole-network activity, generally not documenting the sampling algorithm used (Gritsun et al. 2012). From a practical point of view, given the number of parameters in any model, it is unrealistic to expect precise quantitative comparisons in any event.

Generally speaking, as connectivity developed, bursts started at low intensity and long duration, as shown in Fig. 8. As bursts grew more intense and briefer, the net result was that the number of spikes in each burst stayed surprisingly constant (Fig. 10). This is consistent with the latter phase of development observed in experiments using dissociated rat cortex cells cultured on planar multi-electrode plates, which demonstrated age dependent network burst patterns (van Pelt et al. 2004b, a). In general, those experiments showed that:

• Initially short and slightly skewed bursts lasting on the order of 1–2 s evolved, during the 3rd week in vitro (WIV), into long-lasting bursts of about 6s with almost symmetrical firing rate profiles.

• At later ages (about 4th WIV), network bursts tightened up to about 200 ms. Bursts at this age were characterized by highly synchronized onsets, reaching peak firing levels within less than 60 ms—this pattern persisted for the rest of the culture period.

Note that those previous results used a different definition of burst onset and offset, based relative to each burst’s peak firing rate rather than determined to segment each burst from the background interburst activity. This could explain some of the deviation between those and the current results.

Another possible cause for this difference could be the fact that, in the living preparation, neocortical cell cultures pass through a period of delayed development of synaptic inhibition relative to excitation (Ramakers et al. 1994; Huizen et al. 1985). Increasing inhibitory feedback later on might have the effect of prolonging the bursts (van Pelt et al. 2004a). Since our model applied the same growth rate to excitatory and inhibitory cells, this might be one of the factors that made a difference. Burst durations at later ages (about 4th WIV) were about 200 ms, which are consistent with our results.

Maximal firing rates reported vary considerably, possibly depending critically on the particulars of the electrode array used (for the living preparations) or algorithms used to subsample simulated activity (for computer models). Maximal firing rates reported during bursts range from around 500 spikes/s [for example, van Pelt et al. (2004a)] to around 5,000 spikes/s [for example, Gritsun et al. (2012)]. Depending on one’s assumptions regarding the number of neurons recorded by each electrode, these results might correspond to a range of 8–100 spikes/second/neuron (ignoring the issue of fraction of neurons participating in any particular burst); our results, with burst peak APNFR of around 40–100 spikes/second/neuron), are consistent with these.

In the current simulations, there is a definite region of parameter space that produced sustained, stationary bursting. We did not, however, adjust the distribution of firing rates for endogenously active cells as we changed the target firing rate. It is possible that the behaviors seen for the lowest target firing rate (0.1 Hz) would have been greatly altered if we had made the same factor of 10 adjustment in endogenously active cell firing (in that case, considering endogenously active cell firing rate as relative to target rate, rather than in absolute terms). As it stood, almost all endogenously active cells were firing faster than the target rate, and thus, their connectivity to the rest of the network was all but extinguished for a target rate of 0.1 Hz. Thus, at this lowest rate, bursts were still growing more intense and briefer at the end of the simulations, leading in the case of (0.98, 0.1) to global bursting likely the result of hysteresis-based switching between two stable firing regimes—one fast and one slow—as connectivity grew and ebbed.

It is interesting to compare these results with theoretical and experimental analysis of bursting in randomly connected networks with endogenously active cells (Latham et al. 2000a, b). Such analysis indicated that the number of endogenously active cells changed network behavior among low firing rate, bursting, and high firing rate. Changes in connectivity for the (0.98, 0.1) case was equivalent to varying endogenously active cell input to the rest of the network, switching it between low and high firing rate.

The effect of inhibition on burst shape was similarly most pronounced at the extreme value in which 20 % of the cells were inhibitory. Except for the lowest target rate, this led to wider, less intense, more frequent bursts that appeared to be continuing toward extinction. The interplay between this behavior and connectivity is discussed later in this paper.

5.2 Burst timing

Both inhibition and target rate had clear effects on burst timing. Target rate had a very strong effect on interburst interval distribution, as shown in Fig. 13 and Table 3. Lower target rate induced higher $\xi$ value (longer tail), higher $\sigma$ value (more irregular), and higher $\mu$ value (longer IBIs; also seen in Fig. 11 as fewer bursts per unit time). The values of the GEV parameters were in the range of those from living preparations (Gritsun et al. 2011). Because the number of spikes per burst was relatively constant (Fig. 10), these changes meant that rate of burst production was the primary mechanism for network adaptation toward the target firing rate.

Similarly, at the lowest fraction of excitatory cells (0.80), less intense bursts with fewer spikes per burst were produced. Thus, the primary mechanism for network adaptation toward the firing rate was also rate of burst production.

Though the GEV scale values ($\sigma$, Table 3) show little systematic effect, the coefficient of variation (CV, $\sigma /\mu$, Table 2) clearly indicated that bursts were relatively more irregular as inhibition increased and target rate decreased.

5.3 Connectivity

The basic plots, Fig. 6, show individually the common elements of interconnection evolution during network growth. These are summarized in Fig. 7, which also illustrates a clear dependency on simulation parameters.

Overall, radii of all neurons grew until there was sufficient overlap for each cell’s net input to perturb its growth. At that point, the most striking simulation feature is the divergence of growth rates of inhibitory versus excitatory cells. In all but the simulations with greatest inhibition, excitatory cell radii continued to grow, while inhibitory cells’ radii on average decreased. This is in part consistent with smaller (36 neuron) simulations of neurite outgrowth that showed a transient overshoot in connectivity (van Ooyen et al. 1995). However, in that other work, there was an overshoot in all connectivity, not just inhibitory.

The most obvious parameter dependency of connectivity is what might be a bifurcation in parameter space lying between 80 and 90 % excitatory neurons. For simulations on one side of the possible bifurcation point (fraction of excitatory cells = 0.80), inhibitory and excitatory cells’ radii diverged but inhibitory cells’ radii were, on average, greater than those of excitatory cells. On the other side (fraction of excitatory cells$\,=\,\{0.90, 0.98\}$), inhibitory cells’ radii were, on average, less than those of excitatory cells.

It makes sense that less bursting would translate into reduced neuron activity and thus correlate with wider connectivity. What remains unexplained is why this affected, for the most part, only the inhibitory cells or in which direction the causal relationship worked (i.e., does less bursting cause wider connectivity or vice versa). In addition, in those simulations, there was no transient overshoot and all radii were still increasing at the end of the simulation. This may be partially at variance with much smaller outgrowth simulations, in which stronger inhibition (via strengthened synapses, not larger numbers of inhibitory cells) increased overshoot (van Ooyen et al. 1995; van Ooyen and van Pelt 1996), though even in those simulations, higher inhibition levels would cause indefinite radii increase.

Given the rough parameter space exploration here, the quantitative relationship between simulation parameters and connectivity can only be outlined. Tables 4 and 5 present the final mean excitatory and inhibitory radii. Generally speaking, for excitatory cells, connectivity grew shorter range as target rate increased. However, the change was not dramatic—nowhere near proportional to the change in target rate. This reinforces the idea that average network firing rate, and that of individual neurons, was primarily mediated by rate of burst production, which indeed is what is shown in Fig. 11.

Table 4 Mean neurite radii of ordinary excitatory neurons at simulation end (excitatory, non-spontaneously active, interior neurons)

Table 5 Mean neurite radii of inhibitory neurons at simulation end

Just as the fraction of excitatory cells imposes a clear qualitative change in network connectivity behavior, the quantitative effect of this parameter is also much clearer. As fraction of excitatory cells increased, connectivity grew shorter range—there does not seem to be any explanation needed other than the obvious lower need for excitatory input. On the other hand, inhibitory cells seemed much more sensitive to the presence or absence of stationary bursting. On one side of the putative bifurcation, the decrease of excitatory cells from 98 to 90 % (fivefold increase in inhibition from 2 to 10 %) decreased inhibitory cell radii around 10 %; crossing that bifurcation, the change to 80 % excitatory cells (twofold increase in inhibition to 20 %) resulted in an enormous increase in inhibitory radii.

Those observations were for average connectivity radii. Tables 6 and 7 summarize the final standard deviations for excitatory and inhibitory cells, respectively (which can also be seen in Fig. 7). While higher target rates led in some cases to more variability in connectivity for excitatory cells, this relationship was not strong. The effect of changing inhibition on inhibitory cell radii variability was much clearer, with increasing inhibition leading to greater variability, as in other investigations (van Ooyen et al. 1995). Remember that the (0.98, 0.1) simulation didn’t result in bursting at all.

Table 6 Standard deviation of neurite radii of ordinary excitatory neurons at simulation end (excitatory, non-spontaneously active, interior neurons)

Table 7 Standard deviation of neurite radii of inhibitory neurons at simulation end

5.4 Interplay of connectivity and bursting behavior

We can draw some tentative conclusions regarding the interplay between connectivity and bursting behavior. With higher target rates, connectivity grew (arguably) slightly narrower while IBIs grew more regular. Greater inhibition produced clearly more irregular, wider connectivity and more irregular IBIs. Given that the time scale of outgrowth was greater than that of bursting behavior, we conclude that narrower, more regular connectivity produced more regular IBIs, while wider, irregular connectivity produced more irregular IBIs. It is important to note, however, that the effects of varying the two simulation parameters were not straightforward.

Going forward, it may be profitable to consider this in the context of cluster point processes (Gómez et al. 2005). The timing of bursts can be thought of as a primary process; the timing of individual neurons’ spikes, a subsidiary process. This conceptually decomposes the network’s activity into two parts: the timing of the bursts and the timing of individual cells’ spikes within each burst. As cells’ connectivity radii increased, they integrated a larger number of other cells’ spike trains. This pooling reduced the net variability induced in each neuron’s behavior due to inter-neuron (subsidiary process) variability arising from their independent noise and internal parameter differences. The result was greater variability in the observed primary process, which is merely the collective behavior of the entire network. Understanding how different poolings of subsidiary process events in a recurrent network can lead to variability in primary process events—how bursting emerges from the interactions of a huge number of neurons and synapses—will require an examination of the spatio-temporal activity of the entire network to compare activity at the time scales of spiking and bursting.

Appendix: Neuron and synapse parameters

See Tables 8 and 9.

Table 8 Neuron parameters

Table 9 Synapse parameters