Models of cortical networks with long-range patchy projections

Abstract

The cortex exhibits an intricate vertical and horizontal architecture, the latter often featuring spatially clustered projection patterns, so-called patches. Many network studies of cortical dynamics ignore such spatial structures and assume purely random wiring. Here, we focus on non-random network structures provided by long-range horizontal (patchy) connections that remain inside the gray matter. We investigate how the spatial arrangement of patchy projections influences global network topology and predict its impact on the activity dynamics of the network. Since neuroanatomical data on horizontal projections is rather sparse, we suggest and compare four candidate scenarios of how patchy connections may be established. To identify a set of characteristic network properties that enables us to pin down the differences between the resulting network models, we employ the framework of stochastic graph theory. We find that patchy projections provide an exceptionally efficient way of wiring, as the resulting networks tend to exhibit small-world properties with significantly reduced wiring costs. Furthermore, the eigenvalue spectra, as well as the structure of common in- and output of the networks suggest that different spatial connectivity patterns support distinct types of activity propagation.

1 Introduction

The prevailing model for studying cortical network dynamics is based on randomly connected neurons (Brunel 2000; Kumar et al. 2008b; Kriener et al. 2008). More and more studies (Mehring et al. 2003; Kumar et al. 2008a; Roudi and Treves 2008; Kriener et al. 2009) take spatial network features into account, but are largely constrained to locally coupled neurons within the range of a cortical column (for an exception see Johansson and Lansner 2007). In reality however, the cortical network exhibits a distinctive three-dimensional structure: in the vertical direction, perpendicular to the surface of cortex, it is composed of several layers with layer-specific connectivity (Thomson and Bannister 2003; Binzegger et al. 2004); while in the horizontal direction, parallel to the surface, a spatially extended system of connections exists, incorporating several interconnected columns. The distance between connected neurons varies from a few micrometers to centimeters (Schüz and Braitenberg 2002; Lewis et al. 2002).

In this study, we focus on the two-dimensional (2D) horizontal structure of the cortex. We consider specific models for networks with distance dependent connectivity, composed of local and additional long-range connections. In several species and cortical areas, the distant synapses are established in ‘patches’ (Gilbert and Wiesel 1983; Kisvarday and Eysel 1992; Binzegger et al. 2007), i.e. individual axon collaterals project to several spatially clustered neurons, as illustrated in Fig. 1(b). Likewise, the combined projection pattern of groups of adjacent neurons is often confined to a limited number of discrete spots in space (Amir et al. 1993; Lewis et al. 2002; Levitt and Lund 2002). The details of this peculiar spatial arrangement, together with its specificity for certain species or cortical regions, are still a matter of debate.

Fig. 1

Schematic illustration of the distance dependent projection types of a cortical pyramidal cell (PC): local connections are shown in red, horizontal long-range projections that remain within the gray matter (GM) in blue. (a) Schematic representing a side view including the white matter (WM) projections in black. (b) Emulated top view representing an extracellular injection of an anterograde tracer into the GM of a flattened cortex, with spatially embedded PCs (black dots) and patchy axonal arborizations. The gray area in the middle represents the stained halo surrounding the injection site, all other gray areas represent patchy projection sites. The red and blue lines indicate the projections of one exemplary PC

The aim of this study is to improve the architecture of cortical network models, based on realistic connectivity patterns adapted from neuroanatomical data. Therefore, we consider networks with both local couplings and long-range links. We analyze five possibilities for the spatial arrangement of distant synapses: four different realizations of patchy projections and one network model with homogeneously distributed synapses. Each of these models reflects specific structural features which are either derived from neuroanatomical findings or are ad hoc modeling assumptions. We compare these networks to each other, as well as to standard network topologies like random or small-world graphs.

Then, on the basis of their structural differences, we aim to derive possible functional consequences. For instance, we seek for indications of different network dynamics or a preferred method of signal propagation, depending on the connectivity pattern. The methods we apply to statistically analyze and compare the various network models are adapted from stochastic graph theory (Albert and Barabási 2002; Newman 2003; Strogatz 2001). To characterize the topology of a network, one can calculate characteristic network properties, such as the average shortest path length, the cluster coefficient, or the eigenvalues of the coupling matrix. In addition, we determine the amount of common in- and output for pairs of nodes. This is an essential property as correlations in network activity are strongly determined by the sizes of the common input pool (Kriener et al. 2008).

Recently, several studies indicated the importance of spatial structural features. For instance, considering spatially embedded neurons leads to distance-dependent conduction delays. Roxin et al. (2005) demonstrated that varying conduction delays significantly affects the dynamical states of regular ring graphs. We arrive at a similar conclusion for biologically inspired 2D networks (Voges and Perrinet 2009). Previously, we have shown that the details of the spatial embedding indeed influence the characteristic network properties (Voges et al. 2007). Likewise, in Voges et al. (unpublished manuscript), we investigate the consequences of the number of long-range versus local connections for several graph-theoretic descriptors. Both Roudi and Treves (2004, 2008) and Koroutchev and Korutcheva (2006) demonstrated important advantages of including features of real cortical networks such as local connectivity for the retrieval of states in associative memory. Moreover, Johansson and Lansner (2007) present and discuss a large columnar network model of cortex that includes remote patchy projections between columns.

More often, long-range (patchy) connections are discussed in relation to wiring costs of biological networks, i.e. the total length of axonal fiber necessary to establish any particular network. Since space and energy are limited resources inside the skull (Attwell and Laughlin 2001), wiring optimization is essential to ensure both fast and efficient signal propagation (Chklovskii 2000; Chklovskii 2004; Buzsaki et al. 2004; DeLosRios and Petermann 2007). Optimal wiring rules provide a high connectivity but use little cable length. This is most likely one function of long-range patchy connections. Furthermore, several studies on the visual system suggest a relation to the orientation selectivity of cortical neurons (Bosking et al. 1997; Gilbert and Wiesel 1989; Buzas et al. 2006; Chisum and Fitzpatrick 2004). Combined with the pinwheel pattern in primary visual cortex (Bonhoeffer and Grinvald 1991, 1993), this relation in turn implies strong constraints on the spatial arrangement of the patchy projections.

In this study, we ask whether patchy connections are an important feature which significantly affects one or more of the characteristic network properties. If so, they should be included in future models of the cortex. However, there is still a lack of data in terms of how exactly these patchy projections should be implemented. Thus, the community relies on reasonable assumptions, at best derived from the little data available. We suggest five possibilities for spatial arrangements of long-range projections (four of which make use of patches), identify their characteristics and investigate the effects of distinct projection patterns. In terms of possible functional consequences, we discuss their efficiency with respect to wiring costs and speculate about their corresponding network dynamics and favored types of activity propagation. The general goal is to develop more realistic but nonetheless tractable cortical network models, and to collect evidence for the most appropriate ones. Thus, we intend to help to improve future studies on network dynamics by not only excluding some structural features, but also suggesting network architectures that more adequately represents cortical wiring.

In the next two sections, we describe the network models to be investigated: in Section 2, we give a qualitative description with a particular focus on the neuroanatomical data from which they are derived, while in Section 3, these models are characterized more quantitatively. In Section 4, we explain the graph theoretical tools we employ to analyze the five network models. We then present the results of calculating the networks’ characteristic properties. Finally, on the basis of these results, we discuss and compare in Section 5 the effects of different spatial arrangements of long-range connections. We describe which models we expect to be optimal in terms of cortical requirements and their relevance with respect to neuroanatomical findings.

2 Models derived from neuroanatomical data

This section describes the network models to be analyzed, as well as their neuroanatomical motivation. We refer to two types of neuroanatomical experiments where a tracer injection is taken up by the neural tissue revealing its complex connectivity patterns. An intracellular tracer injection results in a 2D or 3D reconstruction of a single nerve cell. Such studies show the patchy structure of single axon collaterals in pyramidal neurons (Gilbert and Wiesel 1983; Ojima et al. 1991; Kisvarday and Eysel 1992; Binzegger et al. 2007). More often, extracellular bulk injections are employed (Amir et al. 1993; Lund et al. 1993; Pucak et al. 1996; Schüz et al. 2005). They reveal that groups of neighboring cells often project to a number of common spatial clusters, as indicated in Fig. 1(b). An anterograde (or retrograde) tracer is taken up by thousands of neurons located near the injection site. Then, the tracer is transported from the cell bodies along the axons (or dendrites) to its terminals. Anterograde stainings indicate where the neurons project to, while stainings resulting from a retrograde tracer injection show where they get their input from.

The neuroanatomical findings in such publications lead us to consider the following scenario: we study a 2D square sheet (8 mm side length) of pyramidal cells located within one cortical area, representing a flat projection of the cortical surface (Fig. 1(b)). Each neuron targets both neurons situated within its local neighborhood and remotely located neurons as illustrated in Fig. 1(a). The local connectivity has a range of 0.5 mm (red in Fig. 1), and the intra-areal long-range connections (blue in Fig. 1) reach up to 4 mm. We assume at least 50 % local couplings for each pyramidal cell (Schüz et al. 2005). Furthermore, in case of patchy projections, each cell has 3 patches of 0.5 mm diameter (Kisvarday and Eysel 1992; Binzegger et al. 2007; Ojima et al. 1991). For a detailed explanation of these assumptions see Voges et al. (unpublished manuscript).

Concerning the spatial arrangement of the long-range projections, the picture is less clear. For instance, both Buzas et al. (2006) and Amir et al. (1993) suggest a preferred long-range projection length or a preferred patch size which would be specific for one cortical area of a certain species. A particularly interesting but still unresolved issue concerns the number of common projection targets of different neurons. Inspired by neuroanatomical data, we develop five distinct possibilities for the spatial arrangement of long-range projections. Some of them are focused on single neuron properties, while others pertain to extracellular tracer injections. They differ according to their degree of randomness and with respect to the probability of common projection targets. Figure 2 shows the corresponding projection schemes, each of them characterizing one particular connectivity pattern.

Fig. 2

Schematic figures representing five different possibilities of the spatial arrangement of long-range projections: no patches (NP), random patch positions (RP), overlapping patches of neighboring neurons (OP), shared patches (SP) where all nodes in a box project into three out of six possible patches, and partially overlapping patches (PP) where only three out of six patch positions change from box to box

No patches (NP): In this ‘basic’ model we assume a spatially homogeneous distribution of the distant synaptic targets of each pyramidal cell (PC). It provides the baseline to compare with in order to pin down the effect of spatially clustered connectivity patterns. Moreover, at a first glance, this model represents the findings of van Hooser et al. (2006), who claim the absence of patchy connections in the primary visual cortex of mammals without orientation maps (see also Random patches below and Section 5).

Random patches (RP): Here, we include spatially clustered projection targets. The patch positions are randomly chosen, independently of the spatial position of the source node, and independent of each other. This is the simplest assumption if no further information about the relation between the neurons and their patch positions is available. For this model there is only a small probability that two neurons project onto the same distant target. In terms of neuroanatomical data, it corresponds to Johnson et al. (2000) who state that the PC projections in the piriform cortex of the rat show very little overlap. Later on, we will discuss why the results of van Hooser et al. (2006) are probably better represented by this RP than by the NP model.

Overlapping patches (OP): This rather deterministic model is based on a systematic relationship between all patch positions. It assumes a partial overlap of the termination fields of neighboring neurons. In actual fact, this is a rather unrealistic assumption. Still, the combined projection pattern of a cigar-shaped group of adjacent neurons is similar to the elongated stripes in the monkey prefrontal cortex (Lewis et al. 2002; Melchitzky et al. 2001).

Shared patches (SP): Instead of focussing on single neuron projections, we now consider the collectivity of all synaptic targets of groups of neurons. This model is inspired by neuroanatomical findings dealing with extracellular tracer injections. All neurons in a spatially confined region (indicated by the boxes in Fig. 2) establish a common patchy projection pattern, while each single neuron projects into a subset of all patches. Such models imply an increased probability that adjacent neurons project onto the same distant target as, for example, suggested by Ojima and Takayanagi (2004), Kisvarday and Eysel (1992).

Partially overlapping patches (PP): This model is also based on a common patchy projection pattern for neurons situated in a spatially confined region. Compared to the SP model it provides an additional probability for common synaptic targets of neurons located at a larger distance from each other, i.e. situated in different boxes (see, e.g., Kisvarday and Eysel 1992; Ojima and Takayanagi 2004).

The following section provides a more detailed, quantitative explanation.

3 Quantitative description of the models

We consider network models that consist of N = 3600 nodes, representing neurons with directed synaptic connections. Thus, our cortical network models are constituted by directed graphs G, which are specified by non-symmetric adjacency matrices A(G) = (a _ij ). We set a _ij  = 1 if a link from PC i to j exists, otherwise a _ij  = 0. Multiple synapses for a pair of neurons are not allowed in our models. The synaptic connections are established according to probabilistic rules common to all neurons. As a consequence, the same distribution of incoming (in-degree $P(k_\text{in})$) and outgoing links (out-degree $P(k_\text{out})$) holds for all nodes. In any specific network realization, however, each node has random in- and out-degrees. Likewise, all other network properties assume random values if computed from individual networks. To obtain characteristic mean values, we generated 20 independent realizations for each network model, and calculated the corresponding averages.

In particular, we consider 2D spatially embedded networks. Each neuron is situated in a quadratic domain of extent R = 8 mm wrapped around to a torus to avoid boundary effects, see Fig. 3(a). Therefore, the maximum radius of a non-overlapping circular neighborhood is $r_\text{max}=4$ mm. The positions of all nodes are drawn independently and identically from a uniform probability distribution. We distinguish between two separate types of connections (Fig. 3): the local couplings, where each node is connected to all nodes within a distance of $r \leq r_\text{loc}=0.5$ mm with uniform probability $p_\text{loc}$ (red in Fig. 3), and long-range connections, which are synapses established between nodes at distances $r_\text{loc} < r < r_\text{max}$ (blue in Fig. 3).

Fig. 3

Schematic model of spatially embedded neurons with separate local (red) and long-range (blue) connectivity, surface view of a 2D sheet of cortex. Projections shown for one exemplary neuron, represented as the black dot in the center. (a) Homogeneous distal projections, synapses are represented by open blue circles. (b) Clustered long-range connectivity, filled cyan disks represent patches. Their size is characterized by the patch radius $r_\text{p}$. Spatial positions relative to the projecting cell are given by a radial distance $d_\text{p}$ and an angle Φ

Throughout all networks studied here, we consider a constant global connectivity $c=\bar{k}/N=0.0123$ which leads to a mean in-degree (and out-degree) of $\bar{k} \approx 44.1$. This global connectivity is composed of separate local and long-range terms $c=c_\text{loc}+c_\text{lr}$, where

$$ c_\text{loc} = p_\text{loc} \pi r_\text{loc}^2 / R^2 \label{eq3} $$

(1)

Assuming that all synapses are locally established this equation yields c = 0.0123 for $p_\text{loc}=1$. The ratio between the number of local versus the number of long-range links enters our model as a parameter. We consider networks with 50 % up to 100 % local connections. (Specifically, the values $p_\text{loc}= 0.5,0.6,0.7,0.8,$ 0.9,0.95,0.98,0.99,1 are used.)

As indicated in Fig. 3, we distinguish two different possibilities for the spatial arrangement of the long-range projections. For the no patches (NP) model we assume a homogeneous connection probability for all nodes located within $r_\text{loc} < r <r_\text{max}$ (Fig. 3(a)). This equates to randomly selected distant projection targets. Such networks are constructed in a similar way as small-world networks (Newman 2003; Strogatz 2001). The latter are based on a regular ring graph with neighborhood couplings. In a so-called rewiring procedure, the local links are replaced by randomly selected new ones. For a rewiring probability ϕ = 1, one obtains a purely random graph.

Regarding the other four models, we construct spatially clustered projection patterns. Here, we impose additional restrictions on the selection process. The long-range links of each node are confined to a small number of localized termination fields or patches (Fig. 3(b)). For each PC we assume three patches of radius $r_p=r_\text{loc}/2$. The position of each of these patches is parameterized by its radial distance d _p to the presynaptic node and an angle Φ (Fig. 3(b)). To be compatible with periodic boundary conditions, and to avoid overlap between local and patchy connections, d _p is confined to $r_\text{loc}+r_\text{p} \leq d_p < r_\text{max}-r_\text{loc}$ (although $r_\text{max}-r_\text{p}$ would have been sufficient).

The four patchy network models we analyze in this study were introduced and motivated in Section 2. In the following, we give a quantitative parametric description of these networks. Figure 4 shows exemplary representations of the corresponding projection patterns.

Fig. 4

Exemplary realizations of the five network models characterized in Fig. 2. The gray dots in the (x,y) plane show the spatial positions of the nodes, the colored symbols indicate their connectivity patterns. In the upper row, red squares represent locally established synapses, while blue, light blue, and cyan squares represent long-range projection targets of either three selected nodes (black stars in the NP, RP and OP model) or for all neurons in a box (SP and PP model). In the bottom row, each plot depicts two simulated tracer injections (0.5 mm diameter) in the corresponding network model: green stars indicate the injection sites, blue symbols represent an anterograde tracer (postsynaptic projection targets), and red symbols a retrograde tracer injection (presynaptic neurons)

Random patches (RP): For each node, the patch positions are randomly drawn from a uniform probability distribution.

Overlapping patches (OP): Due to nearly identical patch parameters d _p and Φ for each node relative to its position, the termination fields of neighboring neurons partly overlap. We choose d _p  = 2.5 ±0.0625 mm and Φ = nπ/3 ±π/20 for n = 1,2,3 where the given ranges indicate small, again uniform variations we permit.

Shared patches (SP): The 2D plane is subdivided into 16×16 non-overlapping boxes of side length 0.5 mm. For each box six patch positions are randomly selected. Each node is forced to project to three out of the six patches assigned to its box, resulting in shared termination fields. In case of patches determined for boxes, d _p is restricted more strongly to $2.5\,r_\text{loc} \leq d_p <$$r_\text{max}-2\,r_\text{loc}$.

Partially overlapping patches (PP): Going from one box to its neighbor in the x-direction, three patches are maintained, while for the others d _p and Φ are changed. In the y-direction, the patches are again independently selected.

The upper row of Fig. 4 shows an exemplary realization of the nodes’ projections for each of the five network models. For the SP and the PP model, all links of all nodes in two neighboring boxes are shown, resulting in 12 and 9 patch positions for two boxes, respectively. To build a bridge between these theoretical network models and ‘real-world’ neuroanatomical data we simulated tracer injections. The resulting plots are presented in the bottom row of Fig. 4. One node in a network is randomly selected. Then, in case of simulating an anterograde tracer injection, the projection targets of all nodes located at a radial distance r ≤ 0.25 mm (corresponding to an injection site of 0.5 mm diameter) are marked by blue symbols. The red symbols indicate a retrograde injection which is always applied to one (blue) region of the postsynaptic projection targets. Each plot in Fig. 4 demonstrates one network model where both tracer types are injected at two different sites.

4 Statistical network analysis

In the following we first describe the tools of graph theory we use to analyze the network models presented in the previous sections and then present the results of calculating the characteristic network properties. On the basis of these results, we compare the effects of the differences in the spatial arrangement of long-range connections.

4.1 Methods of statistical analysis

The following descriptors are used to characterize and compare the different network models (Albert and Barabási 2002; Newman 2003): (1) Histograms of the numbers of incoming or outgoing links for all nodes of a graph give an estimate of the distribution of in-degrees $P(k_\text{in})$ and out-degrees $P(k_\text{out})$, respectively. (2) The shortest path L _ij is the minimal number of hops necessary to get from node i to node j respecting link directions. We consider here the average shortest path length $L = \frac{1}{N(N-1)} \sum_{i \neq j} L_{ij}$ for all pairs of distinct nodes, referred to as ‘characteristic’ path length. (3) The cluster coefficient C _i is the fraction of all potential links established between any two nodes receiving a link from node i. Again, we consider the mean cluster coefficient $C = \frac{1}{N} \sum_i C_i$. (4) For any graph G with N nodes, we numerically determine the N (complex) eigenvalues λ of its adjacency matrix A(G) and estimate the eigenvalue density P(λ) based on 20 samples of graphs of the same type. (5) The corresponding eigenvectors v of A(G) are also numerically determined (Albert and Barabási 2002). To quantify the spatial spread of a normalized eigenvector v, we use the weighted 2D circular variance

$$ V = 4 - 2 \Bigl| \sum\limits_k |v_k|^2\,e^{2 \pi i x_k / R} \Bigr| - 2 \Bigl| \sum\limits_k |v_k|^2\,e^{2 \pi i y_k / R} \Bigr| \label{eq1} $$

(2)

where v _k are the components of v satisfying $\sum_k v_k^2 = 1$, and (x _k ,y _k ) denotes the spatial coordinates of node k. Complex numbers are used here to conveniently account for the fact that the neurons in our model are arranged on a torus. The circular mean (Batschelet 1981; Fisher 1993) of x-coordinates across all neurons $\mu_c = \sum_k e^{2 \pi i x_k / R}$ is used to obtain the average x-coordinate in a consistent manner. The circular variance $\sigma^2_c = 2(1-|\mu_c|)$ provides a measure of dispersion of the x-coordinates, small values of $\sigma^2_c$ indicate a high concentration on the circle. For any eigenvector v, we consider here the sum of the circular variances for x- and y-coordinates, respectively, each weighted according to the “participation” of individual nodes k given by the coefficient $|v_k|^2$. This definition gives values for 0 ≤ V ≤ 4. Small values of V indicate that the ‘mass’ encoded by the squared components of v is concentrated in a compact spatial region, while larger values of V imply that it is more uniformly spread over the whole domain. Similar results can be obtained by calculating the entropy of the spatial distribution of the v (Voges et al. unpublished manuscript). (6) The Euclidean distance D _ij between two nodes i and j is the length of the shortest cable or connection necessary to establish a physical link. The total pairwise distance D = ∑ _i ≠ j a _ij D _ij is a measure of the total cable length that is necessary to realize a given network. In addition, to account for common axonal projections in patchy arrangements we calculate a modified wiring length Dp: for the long-range links of every node we evaluate the spatial distances between their targets $D_{i_1,i_2}$. If they are located near each other ($D_{j_1,j_2}< 2\,r_p$) we assume they belong to a common patch (each target of each node may only belong to a single patch). For a patch containing n _p projection targets we assume only one long-range link amounting to a cable length $D_{i,j_1}$, plus an additional cable length of (n _p  − 1) r _p to account for patch-local connections. Thus, Dp approximates the wiring length of patchy connections.

Additionally, in order to obtain a quantitative measurement for the mutual overlap of the pre- and postsynaptic populations, we calculate the pairwise similarity in the input and output of every pair of nodes. For any pair of nodes (k,l) the number of common input and output nodes, respectively, is given by the inner product of the associated columns/rows of the adjacency matrix

$$ \begin{array}{lll} &&{\kern-6pt} \textrm{common input:} \quad \sum\limits_i a_{ik} \cdot a_{il},\qquad\\ &&{\kern-6pt} \textrm{common output:} \quad \sum\limits_j a_{kj} \cdot a_{lj}. \label{eq2} \end{array} $$

(3)

In addition, we cluster the nodes in any particular network according to this pairwise similarity measure, i.e. the number of common in- or output nodes. If the in- or output similarity of a pair of nodes is greater than a given threshold θ, the pair is considered to belong to a cluster. Thus, a cluster for a certain threshold is defined by the corresponding amount of pairwise common input or output connections, respectively.

4.2 Results of statistical analysis

We now compare the results of the statistical analysis of the five network models presented in Section 2, based on numerical experiments. The motivation is to identify which properties of networks with patchy connections offer an interesting alternative or even make them a requirement in studying cortical dynamics. Another aim of this work is to substantiate the distinctions of networks with different long-range connection patterns and to identify possible consequences for the corresponding activity propagation.

Due to random node positions, the in-degree distributions are binomial (Fig. 5(a)), independent of the network model and the specific value for $p_\text{loc}$ (Voges et al. 2007). Likewise, $P(k_\text{out})$ for the NP, RP, and OP models is approximately binomial. However, as shown in Fig. 5(b), for the SP and the PP models in case of $p_\text{loc}<1$, the out-degree distribution deviates from a binomial case: with an increasing number of long-range links, $P(k_\text{out})$ becomes broader and flatter. Thus the two network models where the projections of all nodes in a box are confined to certain patch positions have more nodes with exceptionally high or low out-degrees, respectively. The heterogeneity of degrees in a network presumably affects its dynamics (Denker et al. 2004; Tetzlaff et al. 2005, 2009; Kriener et al. 2008): a broad degree distribution enhances fluctuations in the spike rates, and it shapes the power spectrum of the population activity by partially destroying the global oscillations in certain frequency bands. Compared to the other network models, we therefore expect a broader spike rate distribution and reduced global oscillations for the dynamics of the SP and PP model.

Fig. 5

(a) In-degree distribution of all network models for $p_\text{loc}=0.5$ and $p_\text{loc}=1$. (b) Out-degree distribution of the SP and PP network models for $p_\text{loc}=0.5,0.7,1$. In both plots the corresponding binomial distribution (BD) is indicated by black open squares

Moreover, in SP and PP models $\bar{k}_\text{out}$ decreases with decreasing $p_\text{loc}$. The reason is an increase in the probability of the overlap between local and patchy connections, as well as an increase in the overlap between the patches of a single node. In particular, in the example of the PP model with many long-range links, there is an increased chance of hitting an already occupied entry in the adjacency matrix. Generally, the differences in the results for our five network models are largest for the networks with fewest local connections ($p_\text{loc}=0.5$) as our models differ only in their long-range connectivity. This holds for all measurements that will be discussed in this study.

Figure 6 shows the dependency of the cluster coefficient, the characteristic path length and the wiring length on the parameters of the model networks. The panels (a), (b) and (e) prove that all our network models exhibit a strong ‘small-world effect’: a very small number of long-range connections is sufficient to dramatically decrease the characteristic path length L. However, the cluster coefficient C remains relatively high, due to the dominance of local connections (Strogatz 2001; Newman 2003). In Voges et al. (2007) we discussed the small-world effect of spatially embedded networks, but not for patchy connections. The NP model with homogeneously distributed remote connections shows the strongest decrease in L, as well as in C, followed by the RP model, while the OP model exhibits the highest L. Compared to the NP model, the assumption of clustered remote projections gives rise to a weaker reduction of L and C. An explanation is that the targets of a patchy projection are more likely connected to each other. This effect is much smaller than expected, as only very few short-cuts are sufficient to induce a strong reduction of L. A similar argument holds for the effect of overlapping patches. Thus, despite small variations across models, our experiments confirm that patchy projection patterns do not interfere with the small-world property. In Fig. 6(e) the curves representing networks with 70% up to 99% local connections are all above the diagonal, indicating high clustering combined with small characteristic path lengths.

Fig. 6

(a–d) Clustering coefficient C, average shortest path length L, wiring length D, and modified wiring length Dp in dependence of the relative number of local connections $p_\text{loc}$. (e) Small-world property: C plotted versus L. (f) Wiring efficiency: Dp plotted versus L. The dashed lines are the diagonals. All values are normalized to their corresponding maxima: $C_\text{max}=0.59$, $L_\text{max}=7.22$, $D_\text{max}= 241$ m. Different colors indicate the five network models, see legend in (e). Standard deviations over 20 network realizations were too small to be shown

Similarly, the total wiring length D shown in Fig. 6(c) is largely independent of the spatial arrangement of the long-range links. The small reduction for patchy networks is a result of the different spatial constraints in the case of patches (Section 3). However, as shown in panels (d) and (f), the modified wiring length Dp behaves differently: non-local connections decrease the characteristic path length L but increase the total wiring length (red curve in panel f), when it is based on pairwise Euclidean distances. In contrast, assuming a single long-distance link to each group of spatially clustered synapses significantly decreases the wiring length. Thus, patchy networks are wired in a much more efficient way. Figure 6(f) depicts the relation between the topological distance L and the Euclidean distance Dp: to realize a given characteristic path length L only a very small amount of cable is needed, provided patches are used. In such an optimally wired network only a few steps are sufficient to reach any node. We find that all patchy models perform equally in this respect.

The eigenvalue spectrum of the adjacency matrix can be considered as a graph invariant (Farkas et al. 2001). However, different graphs may have similar spectra. The eigenvalues and eigenvectors of the networks’ adjacency matrix A(G) play a distinguished role for the analysis of the (linearized) activity dynamics of the network. We numerically calculated the eigenvalues and eigenvectors of all A(G) in this study but we only present the results for $p_\text{loc}=0.5$. These networks are dominated by long-range links which is the focus of our study. We find remarkable differences among the eigenvalue distributions of the models considered here, see Fig. 7(b–f). As expected, the NP model exhibits a distribution very similar to that of a small-world network (Voges et al. 2007; Farkas et al. 2001). The bulk of eigenvalues λ covers a disk-like structure centered at the origin, while some exist with real parts Re(λ) > 6 and small imaginary parts Im(λ) forming a narrow tail. In addition, there is a single real-valued outlier at λ ≈ 44, which is directly related to the average in- and out-degree of the network (Albert and Barabási 2002; Farkas et al. 2001). Apart from the narrow tail, such a spectrum is typical of a purely random graph. The patchy networks also show a high concentration of eigenvalues around zero (|λ| < 6), but compared to the NP model, they exhibit in all cases larger imaginary parts for most eigenvalues with Re(λ) > 6. The OP model shows the most peculiar eigenvalue distribution: There are some eigenvalues with Im(λ) > 5, even for Re(λ) < 0, and the tail-like structure has several disconnected parts. This distribution is most dissimilar to that of a random graph, most likely due to the quasi-deterministic rules of its generation. Figure 7(a) shows the probability density function (p.d.f.) of the real part of the eigenvalues. Here, the variations between our five models are less pronounced, only the NP model is well separated by a lower and broader p.d.f. similar to that of a random graph. It is followed by the RP model, while the other three are nearly indistinguishable. As the real part of the eigenvalues is informative with respect to the stability of network dynamics (see Section 5), the influence of the spatial arrangement of patches on the system stability is presumably negligible.

Fig. 7

(a) Distributions of the real part of the eigenvalues for $p_\text{loc}=0.5$. Different colors indicate the five network models. (b–f) Eigenvalue distributions in the complex plane for the five network models, each plot is a superposition of 20 individual spectra calculated from 20 independent network realizations with $p_\text{loc}=0.5$

The results regarding the localization of the eigenvectors in dependence of $p_\text{loc}$ are presented in Fig. 8. Considering only the averages $\bar{V}^{1/2}$ (panel a) obtained from all eigenvectors of each network, all models exhibit surprisingly non-localized eigenvectors $V^{1/2} \lesssim 2$. This is even the case for $p_\text{loc}=1$, i.e. if all synapses are local ones. However, panels (b) and (c) indicate that these nearly identical curves are just a result of averaging. In case of $p_\text{loc}=0.5$ the distributions of V ^1/2, shown in panel (b), are clearly separated: The NP model shows the strongest peak for large values, but it also produces the smallest ones. In contrast, the SP and PP models show lower and broader distributions with a peak located at smaller values. Likewise, the data displayed in panel (c) allow a rough distinction between our five models, mainly in the spatial spread of the eigenvectors in dependence of Re(λ). For Re(λ) ≈ 20 some of the eigenvectors of the NP and the RP model are spatially concentrated (V ^1/2 < 1.5), while this is rarely the case for the OP, SP and PP models.

Fig. 8

(a) Spatial spread $\overline{V}^{1/2}$ of all eigenvectors in dependence of the average number of local versus long-range connections. (b) Distribution of V ^1/2 of all eigenvectors for $p_\text{loc}=0.5$, arbitrary units. The inset illustrates a zoomed-in view on the same curves as in the main figure. (c) Spatial spread V ^1/2 in dependence of the real part of the corresponding eigenvalues, again for $p_\text{loc}=0.5$. Different colors indicate the five network models

The last part of our analysis deals with the amount of common in- and output for pairs of nodes, respectively (again, this analysis is only performed for networks generated with $p_\text{loc}=0.5$). In Fig. 9, the upper two rows reveal that the various models are quite distinct with respect to their in- and output similarity. Note that the distributions of common in- and output are generally not identical, due to the fact that the connectivity matrix A(G) is not symmetric. The output distribution results from the construction of the networks, whereas the input distribution is only implicitly constrained. In the NP model the highest number of common projection targets of a pair of nodes is 22, for the RP and OP model there are up to twice as many, while for the SP and PP model up to 70 common targets are possible, see panel (a). There are only two types of input similarity, see Fig. 9(b): first, in the NP and RP model, the common input pool contains at most 25 presynaptic neurons. Secondly, in the OP, SP and PP networks any two nodes can have more than 40 common input nodes.

Fig. 9

Pairwise similarity measure describing the mutual overlap of the postsynaptic (left) and presynaptic (right) populations, respectively. (a, b) Relative frequency P of the number of (a) common output targets and (b) common input nodes, for pairs of nodes (semilogarithmic plots). (c, d) Number of clusters (single-link clustering) obtained by evaluating pairwise similarities in the (c) output and (d) input depending on the imposed threshold θ. Standard deviations are not shown as they are not significant at this scale. (e, f) Mean (solid lines) ± standard deviation (shaded regions) of the clustering threshold θ in dependence of the cluster size. Different colors indicate the five network models

Surprisingly, with respect to the mean in- or output similarity values the differences across our five network models are rather small. These mean values are calculated by averaging across the common input or output, respectively, of every pair of nodes in a particular network. All models have an average output similarity of 0.54±0.0086, which is also the value of the average input similarity for the NP, RP and OP model, while for the SP model the latter is 0.57±0.0022, and for the PP model it is 0.6±0.0004.

These in- or output similarity measures may be relevant to assess favored strategies of signal transmission in the networks. For example, a high amount of common input together with a large number of common outputs can facilitate synchronous signal propagation. In particular, the amount of common input offers important information concerning the correlation analysis of the networks’ activity, see Section 5.

Figure 9(c) and (d) show the results of the clustering procedure with a threshold imposed on the pairwise common input and output, respectively. For very small thresholds θ < 3, all nodes end up in one large cluster as each node shares at least two input or output nodes with at least one other node in the network. In contrast, for large thresholds θ > 70, every cluster contains just one node, as no pair has more than seventy common pre- or postsynaptic nodes. In between these two extremes, the number of clusters in dependence of the threshold is characteristic of the respective network model. They are determined by the similarity measures, the distribution of which is shown in panels (a) and (b). There is only one crossing: For θ ≤ 21 the SP and PP models have more clusters (i.e., less similarity), while for θ > 21 the RP and OP models are more clustered. This analysis confirms the similarity categories discussed above: the NP model with the smallest amount of common in- and output; the RP model with an equally small amount of common input but a medium output similarity; the OP model with a medium output similarity and a comparably large number of common inputs; and the SP and PP models that exhibit both a high input and output similarity.

So far, we only have discussed the number of clusters, but not their size. Figure 9(e) and (f) show the mean clustering threshold $\bar{\theta}$ in dependence of the cluster size, in the range between 2 and 45 nodes. For each network, we collect all clusters of a certain size, independent of the threshold for which they were generated. We compute the average of these θ values, resulting in a cluster size specific $\bar{\theta}$. Therefore, these plots describe the pairwise similarities between the nodes belonging to clusters of different sizes. Analyzing the structural base of a signal propagation by neuronal groups, this information helps to asses the appropriate group size together with the expected number of common projection targets. Generally, $\bar{\theta}$ slightly decreases with increasing cluster size, indicating a reduction of common input and output for the nodes classified in larger clusters. However, for the NP and RP model the pairwise similarities are independent of the cluster size. Besides these cluster size specific features, Fig. 9(e) and (f) validate the in- and output similarity categories discussed above. The NP model in panel (e) is well separated, while the RP and OP model are very alike in this respect. In panel (f), each of the NP, RP and SP models shows its own characteristic clustering threshold, as opposed to the OP and PP models with overlapping standard deviations.

In addition to the analysis described above, we checked whether our findings are still valid under slightly more general conditions. As it was established that the local connection probability smoothly decreases with increasing distance (Hellwig 2000; Kalisman et al. 2003; Stepanyants et al. 2008), we also considered models with an appropriate distance dependence. Instead of assuming a uniform value of $p_\text{loc}$ we used a Gaussian probability profile for the local links (p ₀ = 1 with σ = 0.28, 0.33, 0.4 instead of $p_\text{loc}=0.5, 0.6, 0.7$, respectively). For these modified networks, we observed that the distribution of eigenvalues in the complex plane showed a slightly different shape with a more disk-like distribution around zero than for the networks with uniform local connectivity. For all other characteristic network properties the differences were negligible.

5 Discussion & conclusions

We developed and compared the topology of various types of parametric models for networks with local couplings and additional non-local links. The main new feature of the models is their horizontal connectivity, including patchy projection patterns. Our comparative analysis shows that the spatial arrangement of long-range connectivity indeed has a strong influence on several characteristic network properties. Now, we discuss these results in more detail, particularly with respect to the neuroanatomical studies mentioned in Sections 1 and 2. We also speculate about possible consequences on functional aspects, such as network dynamics or preferred types of signal propagation.

It is to be expected that cortical networks exhibit a small-world topology (Sporns and Zwi 2004; Buzsaki et al. 2004; Bassett and Bullmore 2006) as they are based on highly clustered local connections in combination with few long-range projections. This leads to low characteristic path lengths (Watts and Strogatz 1998; Strogatz 2001; Newman 2003). These features support fast distributed communication: every neuron directly interacts with most of its neighbors, while remotely located targets are reached within a few synaptic steps. In terms of cortical connectivity one has to account for another important constraint: space is limited in the brain. Many studies discuss optimality principles in cortical wiring (Chklovskii 2004, 2000; Kaiser and Hilgetag 2004; Bassett and Bullmore 2006) but up to now, the situation is still unresolved at the level of intrinsic gray-matter connectivity. We found that the assumption of long-range patchy projections does not interfere with the small-world effect. The strong reduction of the necessary cable length in the case of patchy networks suggests that the use of patches could be part of a strategy to minimize the volume used by the cable. In our view, patchy networks are a special realization of a small-world topology with strongly reduced costs of wiring.

The eigenvalue spectrum of the adjacency matrix is a graph invariant. Empirically, it appears that significantly different network topologies give rise to differently shaped eigenvalue spectra (Farkas et al. 2001). All network models considered here have small-world features, although their spectra differ considerably. If a graph describes the couplings between the components of a linear dynamical system, its eigenvalues carry information about the temporal aspects of its dynamical modes. More precisely, in a linear time-invariant system, the real parts of the eigenvalues of the coupling matrix indicate whether the system exhibits stable dynamical behavior (Re(λ) < 0 for all eigenvalues λ). As all our structural spectra have eigenvalues with positive real parts, the dynamical equations must have an additional reset of the membrane potential to guarantee this condition (for networks of integrate-an-fire neurons see, e.g., Kriener et al. 2008). The imaginary part of an eigenvalue indicates oscillatory behavior, with a large |Im(λ)| implying fast oscillations. Our analysis of the eigenvalue spectra is incomplete in the sense that inhibition was not accounted for and synaptic transmission delays were ignored. Nevertheless, it may be anticipated that the specific shapes of the structural spectra are indicative for certain aspects of the network dynamics. In particular, the broad tail-like structure in the spectra of the RP, SP and PP models might indicate the presence of fast oscillations. In principle, if Re(λ) < 0, a large |Re(λ)| is associated with a short lifetime of the corresponding dynamical eigenmode. For the OP model, this means that the eigenmodes corresponding to large Im(λ) and negative Re(λ) are presumably very transient modes and thus of no importance for the network dynamics. In addition, we can make some speculations concerning the global dynamical behavior of our network models. Due to the general similarity of all spectra shown in Fig. 7 (circular structure around zero with a tail of Re(λ) > 0 and a maximum value at Re(λ) ≈ 44), we expect the system dynamics to be quite similar. Thus we presume that our five models exhibit essentially indistinguishable population activities in the ‘idle’ state of the networks. The specific differences in the spectra may come into play in connection with excited states, for example, in terms of the propagation and processing of input signals that represent external stimuli. This view is compartible with the idea that spatially clustered synapses are actually learnt connectivity patterns (Callaway and Katz 1990), i.e. the results of long-term plasticity processes.

For the spatially embedded networks considered here, we also determined some spatial properties of the structural eigensystem, in particular their spatial spread. In accordance with our above discussion, the average values of the spatial spread of all eigenvectors were very similar for all network models considered here (Fig. 8(a)). Therefore, the differences in terms of the population dynamics are presumably rather small. However, there are a few localized eigenmodes for the NP and RP model with real parts between 10 and 20, while all networks with overlapping distant termination fields (OP, SP and PP model) show no such localized modes. An increased probability of common projection targets appears to induce a more global and less local spread of activity. Again, this may indicate that the specific distinctions between our five models become particularly relevant when signals undergo processing.

Shared input and shared output for pairs of nodes in a network is the structural basis for a number of dynamical properties of networks. For instance, Kriener et al. (2008) show that strong common input amplifies synchrony in recurrent networks. Therefore, we expect more synchrony in the network dynamics of the OP, SP and PP models. In particular, the study of propagation of synchronous activity requires an analysis of the transmission between successive groups of neurons. The synfire chain model (Abeles 1991; Diesmann et al. 1999; Kumar et al. 2008a) is used to explore the conditions under which precisely synchronized action potentials propagate from group to group within the cortical network. In order to excite a neuron in a subsequent group one requires a minimal degree of convergence but also enough divergent connections to keep the number of excited neurons large enough. In this context, the number of common projection targets, as well as the number of common inputs, is of great interest. Local cortical networks meet these conditions by means of recurrent connections. In larger networks with only local couplings, however, transmission remains necessarily local. It is tempting to speculate whether patchy connections can provide the anatomical basis for a non-local propagation of synfire chains. This question has been addressed in Voges (2007). Starting with a group defined on the basis of the cluster analysis presented in Fig 9, the respective next group is determined by all other nodes in the network that receive a certain amount of input from the previous group, and so on. The NP model, for example, is not qualified as the long-range connections are not convergent/divergent enough, while in the RP model the number of common input nodes is too small. However, the models with shared or overlapping patches could provide a delicate connectivity pattern, enabling the spread of synchronous activity over longer distances.

A remarkable finding concerning all network structures considered here is that the input and output populations may exhibit different statistical properties, depending on the network architecture. Neither are the in- and out degree distributions (of the SP and the PP model) identical, nor can it be assumed that the average numbers of common in- and output nodes are equal.

Finally, it is necessary to discuss how close our models are to reality. Due to computational limits we embedded only N = 3600 nodes into 16 mm². This is a rather small number compared to approximately 90 000 neurons per mm³ in mouse cortex (Schüz and Braitenberg 2002) – even if we assume that our network resides in one layer (e.g., layer 2/3 with a thickness of roughly 0.3 mm). Likewise, our nodes have approximately $\bar{k}=44$ synapses, while in mouse cortex, there are about 8000 synapses per cell (Schüz and Braitenberg 2002). Despite a neuron density, which falls short of realistic values by a factor of 100, the models are nevertheless adequate to analyze the features of patchy connectivity. The spatial range of the local and the long-range connections generally matches the values given by the literature, and the patch parameters are adapted from neuroanatomical studies (Kisvarday and Eysel 1992; Wallace et al. 1991; Lund et al. 1993). For a more detailed discussion of these numbers see Voges et al. (unpublished manuscript).

An open question is which network models implement a good solution to the wiring problem of the cortex. All models with patchy connections studied here exhibit a small-world topology combined with a high wiring efficiency, but only the SP and PP models offer additional features. They reproduce the pictures of extracellular tracer injections that have been published for larger animals like cats or monkeys (Binzegger et al. 2007; Amir et al. 1993; Pucak et al. 1996), and they support the long-distance spread of synchronized activity. Both projection patterns, resulting from simulated anterograde and retrograde tracer injections, respectively, show a limited number of patches (Fig. 4). However, only the PP model incorporates an increased probability for common projection targets for neurons located at a distance greater than 0.5 mm (Kisvarday and Eysel 1992; Ojima and Takayanagi 2004). Such a feature is most favorable with respect to the idea of preferred connections between neurons with similar functional properties, e.g., orientation selectivity in visual cortex (Bosking et al. 1997; Buzas et al. 2006; Chisum and Fitzpatrick 2004). Combined with the pinwheel assumption for the spatial arrangement of these neurons, a certain connection probability for remotely located neurons is required for synapses between neurons in different pinwheels. It seems that the PP model captures the long-range projection pattern of primary visual cortex of large mammals best.

The RP model leads to spatially clustered postsynaptic sites (retrograde stainings), while the input neurons are uniformly distributed (anterograde stainings). Such a projection pattern matches the horizontal connectivity in the visual cortex (V1) of mammals without orientation maps. Van Hooser et al. (2006) argue that retrograde tracer injections in the V1 of a rodent do not reveal patches. Yet, other studies (e.g., Lohmann and Rörig 1994; Burkhalter and Charles 1990) show that single cell projections, as well as anterograde tracer injections, exhibit a patchy structure in rat V1. Thus, the RP model might be more appropriate if one intends to model cortical networks of rodents.

The OP model, however, is too deterministic compared to neuroanatomical data. Each injection leads to exactly three patches, independent of the injection size, which does not match reality (Amir et al. 1993; Rumberger et al. 2001). Randomizing this parameter would be a desirable feature of an improved model.

In summary, we developed and compared five realizations of spatially embedded networks featuring horizontal long-range connectivity in the cortex. Based on our analysis we conclude that patchy projections represent an optimized scheme of wiring together distant neurons. Compared to well-known network topologies all network models are clearly different from random graphs, but closely related to small-world networks. While our results predict similar population dynamics in the ‘idle’ state of all models considered here, we also found several hints concerning specific differences in the signal propagation depending on the spatial arrangement of the long-range connections. We suggest to include this feature of spatially clustered projection patterns when modeling cortex-like networks, but attention should be paid to the specific realization of these projections. The architecture very likely depends on the species or on the functional requirements of the cortical area.