Abstract

Neural connectivity at the cellular and mesoscopic level appears very specific and is presumed to arise from highly specific developmental mechanisms. However, there are general shared features of connectivity in systems as different as the networks formed by individual neurons in Caenorhabditis elegans or in rat visual cortex and the mesoscopic circuitry of cortical areas in the mouse, macaque, and human brain. In all these systems, connection length distributions have very similar shapes, with an initial large peak and a long flat tail representing the admixture of long-distance connections to mostly short-distance connections. Furthermore, not all potentially possible synapses are formed, and only a fraction of axons (called filling fraction) establish synapses with spatially neighboring neurons. We explored what aspects of these connectivity patterns can be explained simply by random axonal outgrowth. We found that random axonal growth away from the soma can already reproduce the known distance distribution of connections. We also observed that experimentally observed filling fractions can be generated by competition for available space at the target neurons—a model markedly different from previous explanations. These findings may serve as a baseline model for the development of connectivity that can be further refined by more specific mechanisms.

In the wiring length distributions of various neural systems, there is a clear tendency toward low connection lengths, meaning that most neuronal projections are short. Nonetheless, a substantial number of long-distance connections exist, and these connections are important for reducing the number of processing steps in cortical and neuronal networks (Kaiser and Hilgetag 2006). On several scales of neural systems, from the connectivity between individual neurons in Caenorhabditis elegans or the mammalian visual cortex to the connectivity between large-scale cortical regions, the probability of establishing a connection decays exponentially with the distance between neurons or regions.

There are several possible explanations for how this exponential distance dependence may arise. For example, the target of a connection could be genetically encoded by axon guidance molecules (Yamamoto et al. 2002; Borisyuk et al. 2008). Under such a scenario, the target region secretes guidance molecules that diffuse through the tissue. Because there is a higher concentration near the source where the molecule is secreted than at distant regions, a concentration gradient arises, and axons can travel in the direction of higher concentration, toward the target region. The concentration of molecules deposited at a location decays exponentially with time and with distance away from that location (Murray 1990). This relationship is given by 

graphic
where D is the diffusion coefficient and Q is the initial amount of particles per area. Given a threshold for the detection of guidance molecules, neurons closer to the source will be more likely to pick up the signal than neurons further away, leading to distance dependence in the establishment of connections. However, this model would incur a hard border—a distance beyond which the concentration falls below the threshold and where no connections occur, whereas for distances shorter than the critical distance, the probability of establishing a connection would be close to 100%. Under this regime, an inverse sigmoid distribution of connection lengths would be expected, which is unlike the curves that are found in neural systems. In addition, guidance through attraction and repulsion is usually discussed for the global level of long-distance connectivity but not for the local connectivity within regions. Whereas guidance cues are a defining factor at the global level, their role at the local level is limited, due to the problem of establishing a stable gradient for very short distances (less than 0.7 mm). The standard hypothesis for guidance clues is that they are provided by controlled gene expression, specific transmitter–receptor systems and growth factors (Sperry 1963; Yamamoto et al. 2002). This hypothesis involves diffusion-based mechanisms. Recent studies investigated the genetic component of connection development in C. elegans (Kaufman et al. 2006; Baruch et al. 2008) and found that a substantial amount (ca. 40% on average) of the variability of connection patterns in C. elegans cannot be accounted for by differences in gene expression patterns (Kaufman et al. 2006; Baruch et al. 2008).

An alternative hypothesis, called “Peter’s Principle” by Braitenberg and Schuez (1998), suggests that neural outgrowth is basically random, and that specificity in the wiring is derived from the overlap of specific neuronal populations (Binzegger et al. 2004). Here, we explore whether basic mechanisms of random axonal growth can also explain the connection length distributions in neural networks. We focus primarily on local connectivity and assume a simple model with unspecific establishment of connections.

It is well known that axonal growth cones (specialized structures at the end of outgrowing neurites) tend to grow in approximately straight lines unless their way is obstructed or when they encounter attractive or repellent guidance cues in the surrounding medium (Sperry 1963; Easter et al. 1985; Yamamoto et al. 2002). Given a uniform distribution of neurons in a 2-dimensional (2D) space, a straight growing axon is more likely to hit a neuron in its immediate vicinity than further away. If we assume that the axon establishes a connection with the first neuron that it encounters, the probability of establishing connections between neurons would decay exponentially with distance. A further condition that we test is that neurons with which the axon could make a connection are already saturated with other incoming connections. That is, the dendritic tree of the neuron is already occupied with input synapses from other neurons. In that case, the axon would need to continue its growth along the same direction in order to find another candidate target. This aspect of the model introduces the idea that the number of synapses that can be established on a dendrite is limited. This occurs if the dendritic surface area and consequently the number of spines is limited.

Although numerous models for the establishment of neural connections have been formulated (Van Ooyen et al. 1995; van Ooyen and Willshaw 1999; Van Ooyen 2005), and models exist that describe the exponentially decaying connection probability (Amirikian 2005), to our knowledge no model has been presented for the underlying mechanisms by which this distance dependence in connectivity can arise.

In addition to providing a model for distance-dependent connectivity, here, we also show how occupation of neurons introduces competition for connection targets. As a result of such competition, only a fraction of all possible synapses (actual connections) is formed, out of the many potential connections that might occur when an axonal growth cone is spatially close to a dendritic tree (Fig. 1). This so-called filling fraction (Stepanyants et al. 2002) has been measured experimentally and is found to vary with the maximum number of synapses per connection or per neuron. In brief, our model reproduces the widely observed connection length distributions through random neural outgrowth, and provides an explanation of experimentally found filling fractions through the additional factor of neural target competition. Naturally, there are several other topological and spatial network properties (Costa, Kaiser, and Hilgetag 2007; Costa, Rodrigues, et al. 2007; Kaiser et al. 2007) that can be considered, such as characteristic path length, clustering coefficient, or modular organization. In the present study, however, we focus on filling fraction and length distribution, because these comprise network features that are most often reported in anatomical studies. However, when more detailed neuronal connectivity data become available, one can readily compare model and experimental data using a variety of additional connectivity measures.

Figure 1.

Competition for establishing neural connections. (A) An axonal growth cone can establish a synaptic connection with a dendritic spine if there is available space. (B) If all available dendritic spaces are already occupied by competing neurons, the potential synapse will not be established and the axon will continue its search and eventually establish connections with other neurons in its path.

Figure 1.

Competition for establishing neural connections. (A) An axonal growth cone can establish a synaptic connection with a dendritic spine if there is available space. (B) If all available dendritic spaces are already occupied by competing neurons, the potential synapse will not be established and the axon will continue its search and eventually establish connections with other neurons in its path.

Materials and Methods

Neural Networks

Primate Corticocortical Network

We analyzed the spatial arrangement of 2402 projections among 95 cortical areas and subareas of the primate (Macaque monkey) brain. The connectivity data were retrieved from CoCoMac (Kötter 2004; Stephan et al. 2001) and are based on 3 extensive neuroanatomical compilations (Felleman and van Essen 1991; Carmichael and Price 1994; Lewis and Essen 2000) that collectively cover large parts of the cerebral cortex. Spatial positions of cortical areas were estimated from surface parceling using the CARET software (http://brainmap.wustl.edu/caret). The spatial positions of areas were calculated as the average surface coordinate (or center of mass) of the 3D extension of an area (cf. Kaiser and Hilgetag 2004).

Caenorhabditis elegans Neuronal Networks

We further analyzed 2D spatial representations of the global neuronal network (277 neurons, 2105 connections) of the nematode C. elegans, as well as a local subnetwork of neurons within C. elegans rostral ganglia (anterior, dorsal, lateral, ring, 131 neurons; 764 unidirectional connections). Spatial 2D positions (in the lateral plane), representing the position of the soma of individual neurons in C. elegans, were provided by Choe et al. (2004). Neuronal connectivity was obtained from Achacoso and Yamamoto (1992). This compilation is largely based on the data set of White et al. (1986), in which connections were identified by electron microscope reconstructions. Details of the data set are described in Kaiser and Hilgetag (2006). Wiring length was calculated as direct Euclidean distance between connected components in 3D (Macaque) or 2D (C. elegans). Both data sets are available at http://www.biological-networks.org.

Rat Local Network

The data were taken from Lohmann and Rorig (1994). The data set contains connections of supragranular pyramidal neurons (layers II and III) in the rat extrastriate visual cortex. The study measured the length of 467 horizontal axon collaterals in area 18a. Only 3–5 of the longest branches of neurons were examined using light microscopy (Lohmann and Rorig 1994).

Neural Connectivity Simulation

Sparse Neuron Population

We tested a condition in which 400 neurons were randomly arranged on a 100 × 100 unit 2D embedding space. As each cell filled 1 square unit, only 4% of the potential space was filled with neuron bodies. Subsequently, each neuron was randomly assigned one direction for axon outgrowth. The axon was growing in a straight line until another neuron was hit; thereafter, a connection between both neurons was established. The straight axonal growth away from the soma reflects the experimentally observed somatofugal growth pattern of neurites (Samsonovich and Ascoli 2003). The axon continued growing until the borders of the embedding space were reached or the maximum number of established synapses for that neuron had been reached. An axon was considered to be close enough to another neuron for establishing a connection when the Manhattan distance between the axon tip or growth cone and the neuron was smaller than 1 unit. The projection length of the established fiber was given as the Euclidean distance between the 2 connected neurons. The morphology of dendrites was not included in the distance calculation as the model should be as simple and general as possible for all types of neural tissue. Similarly, axon collaterals were not taken into account to ensure that our results were comparable with other modeling studies and to reduce the number of assumptions of the model.

Under what we call the occupied condition, a neuron was considered unable to establish a connection with a target neuron when the target already had another incoming connection. In such cases, the neuron could not accommodate any additional synapses. As in the previous condition, axons were growing in a random direction until they reached a potential target neuron. If that neuron was occupied—that is, when it already had an incoming connection from another neuron—the axon continued to grow along the same direction until a connection with an unoccupied neuron could be established (or the axon left the embedding space for network growth). We used an upper bound of 1 incoming connection per neuron as a default but also tested other values (see Fig. 1 for a schematic overview of competition of incoming connections).

Dense Neuron Population

In the so-called dense condition, 1000 neurons were placed on the 100 × 100 unit field, filling 10% of the space. Again, the axon was growing in a straight line in a randomly chosen direction and a connection was established with the first neuron it encountered along the way. Also in this scenario, we tested both the potentially occupied and unoccupied condition.

Evolution of Neuron Population Size

Under this condition, the simulation started with 1 neuron positioned randomly within the 100 × 100-unit embedding field, and neurons were subsequently added at each time step until the resulting network reached a density of 1000 neurons. The connections of a neuron were established once it was added to the network. Note that connections could only be established to neurons that existed at that time step. Again, we tested both the potentially occupied and unoccupied condition.

Filling Fraction

The filling fraction of a neuron is the number of actually established synapses divided by the number of potential synapses (Stepanyants et al. 2002). Potential synapses can be formed by all axons that are within the reach of a neuron. In our simulation, this meant that they were less than 1 spatial unit away from the center of the neuron. In the following, the filling fraction represents the average value over all neurons as measured at the end of the simulation.

Results

Wiring Length Distributions of Neural Networks

In the primate corticocortical as well as in the rat and C. elegans neuronal network, the reach of connections among cortical areas or neurons quickly decays with distance (Fig. 2A–D). Nonetheless, some connections are not formed between immediate neighboring cortical areas and extend over a considerable distance (primate cortical network, Fig. 2A). This is also true for the local connectivity among neurons within cortical areas (Fig. 2B). A similar distribution emerged for the local connectivity in C. elegans (Fig. 2C). For the global C. elegans network, some connections are almost as long as the entire organism (Fig. 2D). Thus, all neural systems show a tendency toward short-distance projections while still allowing for the existence of long-distance projections.

Figure 2.

Connection lengths in cortical and neuronal networks. Connection length distributions (×) fitted with a Gamma function (see Table 2 for fit coefficients). (A) Macaque cortical network. (B) Rat supragranular neuronal network. (C) Caenorhabditis elegans local network. (D) Caenorhabditis elegans global network.

Figure 2.

Connection lengths in cortical and neuronal networks. Connection length distributions (×) fitted with a Gamma function (see Table 2 for fit coefficients). (A) Macaque cortical network. (B) Rat supragranular neuronal network. (C) Caenorhabditis elegans local network. (D) Caenorhabditis elegans global network.

We fitted the distance distributions with exponential (a·ebx), power law (a·xb), Weibull (a·b·xb1·eaxb), Gaussian (forumla, and Gamma (c·f(x,a,b)) distributions. Note that the Gamma distribution used an additional factor c for scaling the probability density function f. The functions as well as the R2 as a measure of the goodness of the fit are shown in Table 1. Because the observed anatomical distributions were nonsymmetrical, we chose the Gamma distribution for evaluating the anatomical length distributions as well as for the distributions resulting from simulated neural development. The fitting parameters of the anatomical networks for the Gamma distribution are given in Table 2.

Table 1

Performance of different fitting functions for anatomical length distributions

Network R2
 
 Exponential Power-law Weibull Gaussian Gamma 
Macaque cortex, cortical 0.609 0.348 −0.739 0.879 0.960 
Rat visual cortex, neuronal 0.080 0.001 0.024 0.927 0.900 
Caenorhabditis elegans, local neuronal 0.598 0.303 0.305 0.910 0.957 
C. elegans, global neuronal 0.974 0.981 0.979 0.974 0.824 
Network R2
 
 Exponential Power-law Weibull Gaussian Gamma 
Macaque cortex, cortical 0.609 0.348 −0.739 0.879 0.960 
Rat visual cortex, neuronal 0.080 0.001 0.024 0.927 0.900 
Caenorhabditis elegans, local neuronal 0.598 0.303 0.305 0.910 0.957 
C. elegans, global neuronal 0.974 0.981 0.979 0.974 0.824 

Note: The table shows the different distributions and R2 as a measure of the goodness of fit (see Table 2 for the definition of Gamma used here).

Table 2

Gamma fit for different anatomical networks

Gamma fit: P(l) = c × f (l, a, ba b c 
Macaque cortex, cortical network 3.078 6.378 5.783 
Rat visual cortex, neuronal network 6.075 0.079 0.151 
Caenorhabditis elegans, local neuronal network 2.486 0.010 0.009 
C. elegans, global neuronal network 0.541 0.419 0.210 
Gamma fit: P(l) = c × f (l, a, ba b c 
Macaque cortex, cortical network 3.078 6.378 5.783 
Rat visual cortex, neuronal network 6.075 0.079 0.151 
Caenorhabditis elegans, local neuronal network 2.486 0.010 0.009 
C. elegans, global neuronal network 0.541 0.419 0.210 

Note: f denotes the Gamma probability density function with shape parameter a and scaling parameter b. Parameters a, b, and c relate to the relative length frequency P(l), where l is the fiber length in millimeters.

Wiring Length Distributions of Generated Networks

Sparse Neuron Population

On a 100 × 100 unit grid, 400 neurons were randomly positioned. Axons that hit a target, that is, that had a Manhattan distance of less than 1 unit at that growth step, were establishing a connection on the first encounter, irrespective of there already being other incoming connections at the target (Fig. 3A,B). We also tested the condition where targets could not receive connections once they already had an incoming axon (Fig. 3C,D). In this case, we found that the number of established connections was reduced, and connection lengths were longer. In the first condition, connections were on average about 16.4 units long (maximum length: 95 units), whereas in the second condition, connections were on average 22.6 units (maximum length: 114 units) long.

Figure 3.

Spatially sparse population. (A) Network of 400 nodes and 264 unidirectional connections. (B) Distribution of connection lengths (×) with the Gamma fit indicated as a line (a = 1.5; b = 11.1; c = 11.3; R2 = 0.96). (C) Network of 400 nodes with potential occupation and 217 unidirectional connections. (D) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 15.0; c = 12.6; R2 = 0.98).

Figure 3.

Spatially sparse population. (A) Network of 400 nodes and 264 unidirectional connections. (B) Distribution of connection lengths (×) with the Gamma fit indicated as a line (a = 1.5; b = 11.1; c = 11.3; R2 = 0.96). (C) Network of 400 nodes with potential occupation and 217 unidirectional connections. (D) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 15.0; c = 12.6; R2 = 0.98).

Dense Neuron Population

On a 100 × 100 grid, 1000 neurons were randomly positioned. Axons that hit a target established a connection on the first encounter, again irrespective of there already being other incoming connections (Fig. 4A,B). We also tested the condition where targets could be occupied when they already had an incoming axon (Fig. 4C,D). In this case, the number of established axons was reduced and average connection lengths were much longer. In the first condition, connections were on average about 10 units long (maximum length: 102 units), whereas in the second condition, connections were on average as long as 18 units (maximum length: 101 units). Note that in a dense neuronal population, the effect of potential occupation on the average connection length was considerably stronger than in the sparse population.

Figure 4.

Spatially dense population. (A) Network of 1000 nodes and 796 connections. (B) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 6.8; c = 10.5; R2 = 0.97). (C) Network of 1000 nodes with potential blocking and 690 unidirectional connections. (D) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 12.5; c = 11.7; R2 = 0.97).

Figure 4.

Spatially dense population. (A) Network of 1000 nodes and 796 connections. (B) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 6.8; c = 10.5; R2 = 0.97). (C) Network of 1000 nodes with potential blocking and 690 unidirectional connections. (D) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 12.5; c = 11.7; R2 = 0.97).

In the spatially dense neuron population, the number of connections scaled with the number of neurons (i.e., with a factor of 2.5 for an increase from 400 to 1000 neurons). However, the average connection length was lower than for the sparse network. This was due to the fact that in the vicinity of each node, there were more neurons with which a connection could be established.

Evolution of Neuron Population Size

We also tested the case in which the 1000 neurons were added one after another, starting with 1 initial neuron (Fig. 5A,B). Again, we looked at the effect of occupation, where neurons that had already received 1 axon were unavailable for further incoming connections (Fig. 5C,D). The average connection lengths were comparable with the nonevolving network with 1000 nodes with average connection lengths of 9.7 units (no competition) and 18.4 units (competition).

Figure 5.

Evolution of population size. (A) Network of 1000 nodes and 809 connections. (B) Distribution of connection lengths (x) with the Gamma function fit indicated as a line (a = 1.5; b = 6.4; c = 10.0; R2 = 0.97). (C) Network of 1000 nodes with potential occupation and 697 unidirectional connections. (D) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 12.6; c = 12.1; R2 = 0.98).

Figure 5.

Evolution of population size. (A) Network of 1000 nodes and 809 connections. (B) Distribution of connection lengths (x) with the Gamma function fit indicated as a line (a = 1.5; b = 6.4; c = 10.0; R2 = 0.97). (C) Network of 1000 nodes with potential occupation and 697 unidirectional connections. (D) Distribution of connection lengths (x) with the Gamma fit indicated as a line (a = 1.5; b = 12.6; c = 12.1; R2 = 0.98).

For the case with potential occupation, fewer connections were established. This is due to competition for free places, whereby some axons did not successfully establish a connection before reaching the borders of the embedding space. Because some neurons in the axonal pathway would already be occupied, the length of successfully established connections increased. There seemed to be no difference in this respect with the other 2 cases with potential occupation, indicating that competition for free places and neuron density rather than a static versus evolving neuron population influenced connection length distributions. In the following sections, the role of competition on synapse formation is investigated more closely.

Role of Competition for Available Neuronal Targets

The filling fraction is the number of established synapses divided by the number of potential synapses. A potential synapse occurs whenever axon and dendrite are spatially close (Stepanyants et al. 2002). The filling fraction took a value of 1 if all potential synapses were established, which occurred in our simulations without potential occupation. In the simulations that included potential occupation, the filling fraction decreased with increasing numbers of neurons (Fig. 6A). Previous work pointed out that a low filling fraction increases the number of ways in which synapses could be (re)distributed among target spaces, leading to maximum plasticity, while wiring should be as short as possible (Chklovskii 2004). Simulations also showed a decrease in the probability that any 2 neurons were connected (edge density; Fig. 6B) when the total number of neurons increased. For a low number of neurons, the edge density of the whole network differs from the edge density when isolated nodes are not taken into account. This shows that isolated nodes, that is, nodes that neither establish nor receive a synaptic connection, occur frequently when the number of potential targets in the axonal pathway is low. Isolated nodes are rare at later stages of development when the number of neurons is higher.

Figure 6.

Dependence of filling fraction and edge density on the number of neurons. (A) Filling fraction, which is the proportion of successfully established connections after axon–cell contact, depends on the number of neurons (vertical bars indicate the standard deviation while lines show the average over 10 generated networks). (B) The edge density—the number of existing edges in the network (i.e., connected neuron pairs) divided by the number of all possible edges—decreases for larger numbers of neurons (average and standard deviation as in (A); solid line: complete network; dashed line: network without isolated nodes).

Figure 6.

Dependence of filling fraction and edge density on the number of neurons. (A) Filling fraction, which is the proportion of successfully established connections after axon–cell contact, depends on the number of neurons (vertical bars indicate the standard deviation while lines show the average over 10 generated networks). (B) The edge density—the number of existing edges in the network (i.e., connected neuron pairs) divided by the number of all possible edges—decreases for larger numbers of neurons (average and standard deviation as in (A); solid line: complete network; dashed line: network without isolated nodes).

Such a relative decrease in the number of connections occurs both in the whole network and when only nonisolated nodes with at least 1 edge are taken into account. This indicates that with an increasing number of competing neurons, many axons reach the borders of the embedding space without establishing a connection with any other neuron.

For the previous simulations with 1000 neurons, there was no difference in filling fraction, neither between constant or variable cell sizes (see section Robustness of Results) nor between a static or evolving number of neurons (the overall filling fraction remained around 0.26). Note that the number of neurons has a linear relation with the spatial or neuron, density. Therefore, the filling fraction decreases with neuron density. This relation was also found in experimental studies on connectivity in the mouse, the rat hippocampus, and the macaque visual cortex, where the filling fraction ranges from 0.12 to 0.34 (Stepanyants et al. 2002).

If several (here 10) potential places for synapse formation exist on the neuron, axonal growth cones form synapses earlier so that fewer long-distance connections are established (Fig. 7). Because ample opportunities for synapse formation exist, all potential synapses can be established and the filling fraction becomes 1, whereas it is 0.28 with only 1 available place.

Figure 7.

Histogram of connection lengths. Connection lengths were measured for 10 generated networks resulting in the average distributions shown here. (A) Maximally 1 afferent synapse on a neuron. (B) Maximally 10 afferent synapses on a neuron. The longest possible connection in the embedding space would be 140 units long. Note that due to increased competition for free places, there were more long-distance connections, up to 100 units long, in scenario (A), whereas connections in (B) were only up to 20 units long.

Figure 7.

Histogram of connection lengths. Connection lengths were measured for 10 generated networks resulting in the average distributions shown here. (A) Maximally 1 afferent synapse on a neuron. (B) Maximally 10 afferent synapses on a neuron. The longest possible connection in the embedding space would be 140 units long. Note that due to increased competition for free places, there were more long-distance connections, up to 100 units long, in scenario (A), whereas connections in (B) were only up to 20 units long.

A simultaneous increase of synapses and available space leads to a gradual increase in filling fraction with the number of synapses or places, from 0.28 for 1 synapse to 0.47 for 8 synapses (diagonal of Fig. 8). Each neuron is growing and establishing synapses until the maximum number of outgrowing synapses for that neuron or the borders of the embedding space have been reached. We also tested scenarios where the number of outgoing synapses and of available targets differed (Fig. 8): Having more outgoing synapses than free places led to more competition and therefore to a lower filling fraction (<0.2). More free places than outgoing synapses, on the other hand, reduced competition and led to a higher filling fraction (>0.6). Testing different scenarios showed that the filling fraction f depended on the competition factor r (ratio between outgoing synapses and maximum free places) following an exponential decay, as shown in Figure 9.

Figure 8.

Variation of axon collaterals (outgoing synapses) and available places for synapse establishment. The median filling fraction of 20 generated networks with 1000 neurons for each parameter pair is indicated by the gray level. The maximum number of outgoing synapses denotes how many connections a single neuron can maximally establish. The number of maximum free places for synapses shows how many places each neuron has available for incoming connections.

Figure 8.

Variation of axon collaterals (outgoing synapses) and available places for synapse establishment. The median filling fraction of 20 generated networks with 1000 neurons for each parameter pair is indicated by the gray level. The maximum number of outgoing synapses denotes how many connections a single neuron can maximally establish. The number of maximum free places for synapses shows how many places each neuron has available for incoming connections.

Figure 9.

Influence of competition on the filling fraction. The x-axis shows the ratio between outgoing axon collaterals and maximally available places at potential target neurons of Figure 8. The filling fraction decays with this ratio and follows an exponential function f(r) = 1.125 exp(−r), R2 = 0.91 (solid line).

Figure 9.

Influence of competition on the filling fraction. The x-axis shows the ratio between outgoing axon collaterals and maximally available places at potential target neurons of Figure 8. The filling fraction decays with this ratio and follows an exponential function f(r) = 1.125 exp(−r), R2 = 0.91 (solid line).

Robustness of Results

Variable cell size. In the previous simulations, the size of cells was homogenous with a diameter of 1 unit, but we also tested variability in cell size. For the variable-cell–size condition, the size was randomly chosen from a uniform distribution (0; 2), resulting in an average size of 1 unit. We found that these individual differences in cell size did not lead to any significant changes in filling fraction or connection length distribution (see Supplementary Fig. S1).

Growth in 3D. In the results presented so far, we have focused on 2D growth, which reflects the situation for in vitro slice experiments or in vivo growth where one dimension is physically absent. However, we also tested growth in 3D, which led to a similar shape of connection length distributions and similar filling fractions (see Supplementary Figs S2 and S3).

Discussion

We have tested 3 scenarios of synapse formation during cortical development: 1) axon growth in a spatially sparse versus a spatially dense neuronal population, 2) a gradual increase in the number of neurons versus all neurons present from the outset of development, and 3) having a limit on how many incoming connections a neuron can receive versus a potentially unlimited number of incoming connections.

In all modeled scenarios, the frequency of connections decayed with the distance between neurons. This decay with distance could be fitted with a Gamma distribution, with an excellent goodness of fit (adjusted R2 ≥ 0.98). In analogy, experimental data for the anatomical networks have shown a closely related Gaussian distribution (Hellwig 2000). Furthermore, whereas the gradual increase in the number of neurons (scenario 2) led to fewer connections in the network, the connection lengths increased because some neurons were unavailable for accommodating incoming connections.

We also introduced the idea that the filling fraction—the proportion of established synapses relative to all possible synapses—may arise from neural competition for axonal space on the dendritic tree, rather than being the result of a hit-and-miss model of axon growth. Our model can reproduce filling fractions that were experimentally found in anatomical networks and leads to testable predictions for the development of network connectivity. For example, a decrease in axon collaterals or an increase in space on target neurons should reduce competition and therefore lead to higher filling fractions. Conversely, smaller dendritic trees, with the neuron density remaining the same, should lead to reduced filling fractions. Indeed, a relationship has been found between the size of the dendritic tree and the number of innervating axons that survive into adulthood (Purves and Lichtman 1980), pointing to competition for space in the developmental pruning of axonal connections.

Data Choice

We used anatomical data from the global level of cortical fiber tracts down to the local level of neuronal connectivity in the rat visual cortex and C. elegans. These data sets are the best available but not the best imaginable data sets: The primate corticocortical network reflects macroscopic connectivity, whereas our model concerns microscopic network growth. Caenorhabditis elegans is a system affected by both small size and a ganglionar organization, which is different from the 3- or 6-layer laminar organization of other neural systems. The rat data set, the best proxy of local neuronal networks, is incomplete as only some layers and only the longest branches of connections are reported. However, other more relevant data sets were unavailable to us and the data sets used in this study represent the best data sets we could find. Such limitations of incomplete data sets are typical of local anatomical studies, and it would be very useful for future studies if different data sets at the local level could be combined in similar ways as cortical fiber tract data have been combined (Stephan et al. 2000, 2001; Kötter 2004).

Network Topology

Although the connection length distributions in the generated networks were similar to those found in anatomical networks, modeled and biological networks differed in their topological organization. We tested whether the generated networks showed features of small-world networks (Watts and Strogatz 1998), as found previously in neural connectivity as diverse as the neuronal network of C. elegans and the cortical networks of the cat and macaque brain (Hilgetag et al. 2000). These networks show a low characteristic path length that is comparable with that in a randomly organized network. The characteristic path length denotes the average shortest number of connections that have to be crossed in order to go from one neuron or cortical area to another. At the same time, the clustering coefficient, denoting how well direct neighbors of a node are on average connected, is much higher than for random networks (for a general overview of network features of neural systems, see Sporns et al. 2004). In our simulations, the characteristic path length of the generated networks was below the one for random networks in 5 of the 6 scenarios (it was higher for the nonevolving dense matrix without potential occupation), which means that nodes of the networks can be reached in fewer steps than in a comparable randomly organized network. The clustering coefficient, however, was above the value for random networks in all cases. Additional mechanisms must therefore account for the observed small-world and multiple-cluster organization of biological neural systems. One such mechanism is the occurrence of time sequences and time windows during development: Some parts of the network will develop earlier than others, and the connections will preferentially be established with neurons that are in the same time window for synaptogenesis (Kaiser and Hilgetag 2007; Nisbach and Kaiser 2007). Examples would be the time course for the establishment of layers within cortical areas or time courses of synaptogenesis and neuron migration (Rakic 2002).

We could also have considered additional spatial and topological network properties. Such measures include the degree distribution (Kaiser et al. 2007), the spatial layout of neurons (Kaiser and Hilgetag 2006), and various other network properties (Costa, Kaiser, and Hilgetag 2007; Costa, Rodrigues, et al. 2007). However, opportunities for comparison with experimental data are limited because of the limited knowledge of network connectivity at the local neuronal level. Future studies might thus show that our model agrees with some properties of neuronal networks but not with others.

Model Limitations

The present study uses a simple model that ignores many details of neural development in order to establish what aspects of connectivity may be explained by a minimal mechanism of random axonal growth. If general features of connectivity, such as the typically observed wiring length distribution, can already be explained by random growth, one may conclude that they do not require intricate developmental mechanisms—for instance, axon guidance signaling between pre- and postsynaptic neurons. Such a finding then shifts the focus of research to connectivity features for which more specific mechanisms are indeed required.

Clearly, several aspects of connectivity formation are not random, as demonstrated by the orderly collection of axonal fibers into nerve fiber tracts. The intricate vertical and laminar organization of cortical connections may also be shaped by specific factors, including patterns of gene expression (Tarui et al. 2005), which might affect connectivity formation. Moreover, the geometrical shapes of dendritic and axonal arborizations, with their characteristic curvature, branch angles, tree asymmetry, fractal dimensions, etc. (for a review, see Uylings and van Pelt 2002) may also be important for determining connectivity patterns and space filling (Wen and Chklovskii 2008). It is also likely that there are differences between species (Easter et al. 1985) as well as between spatial scales—for example, the formation of connections between individual neurons or between large-scale cortical areas. Although the assembly of mesoscopic connections may proceed largely in the absence of neural activity (Verhage et al. 2000), synaptic connectivity of intrinsic connections is adjusted via activity-dependent plasticity (Butz et al. 2009). Nonetheless, as our model demonstrates, basic features of connectivity could be formed by random growth processes at the cellular as well as systems level.

Random Outgrowth and Competition for Space

In our model, we assumed a random outgrowth of axons. This assumption is in accordance with experimental results showing that synapse formation, within a specified class of target neurons, can be described as a random process. For example, the distribution of synaptic boutons on a dendrite as well as on pyramidal cell collaterals can be modeled by a Poisson process (Hellwig et al. 1994). In addition, the connectivity pattern of layer-III pyramidal neurons in cat and monkey visual cortex agrees with the hypothesis of random wiring (Kisvarday et al. 1986; McGuire et al. 1991).

We also assumed that neurons might become unavailable for receiving further connections once they already possess a sufficient number of incoming connections. This mechanism could provide an explanation for the experimental observation that axons and dendrites may sometimes be spatially adjacent without establishing synaptic contacts. For example, confocal microscopy and recordings in L5 pyramidal neurons of rat somatosensory cortex have shown that less than one-quarter of touches between connected neurons led to the formation of a synapse. In total, only 10% of all possible synapses (filling fraction, see Stepanyants et al. 2002) for adjacent axons and dendrites were established (Kalisman et al. 2005). In cases where a connection cannot be established, axons will continue to grow beyond the spatially nearby neuron to establish connections with neurons further away.

During the development of the nervous system, cells are commonly innervated by more axons than they ultimately maintain into adulthood (Purves and Lichtman 1980; Rakic et al. 1986; LaMantia and Rakic 1990; Lohof et al. 1996; Van Ooyen 2001). This is a widespread phenomenon in the developing nervous system and occurs, for example, in the formation of ocular dominance layers and columns in the lateral geniculate nucleus and visual cortex (e.g., Wiesel 1982), the climbing fiber innervation of Purkinje cells (Crepel 1982), the innervation of sympathetic and parasympathetic ganglion cells (Purves and Lichtman 1980), and the development of connections between motor neurons and muscle fibers (e.g., Sanes and Lichtman 1999). Importantly for our model, the overproduced axons become eliminated by competitive interactions between the innervating fibers (Rakic 1981, 1986; Rakic and Riley 1983) and may involve competition for target-derived neurotrophic factors (Cabelli et al. 1995). Interestingly, also the cellular geometry appears to play an important role. In many types of neurons, a positive correlation exists between the size of the dendritic tree and the number of innervating axons surviving into adulthood (Hume and Purves 1981; see also Van Ooyen et al. 2000).

Other Models for Filling Fraction

A model for the filling fraction was first described by Stepanyants et al. (2002). In this model, nonestablishment of synapses was explained by axons running close to the dendritic tree of a neuron but missing the dendritic spine in their straightforward movement. However, whereas this model numerically reproduced the experimentally found filling fraction, the explanation may not be biologically realistic: Filopodia of axonal growth cones can be up to 40 μm in length (Van Ooyen 2003), potentially covering up to 80 μm in the forward direction searching for potential binding places. The distance between spines on the dendritic tree, however, is significantly smaller than this search space, with typically 2 spines per micrometer for dendritic segments of cortical pyramidal cells (Braitenberg and Schuez 1998). Competition might provide an alternative explanation for the experimentally observed filling fractions.

Comparison with Local Cortical Connectivity

If some aspects of neuronal connectivity at the local level can result from nonspecific axonal outgrowth, as suggested by the present model, we would predict for neurons that develop early during development that they 1) establish relatively more short-distance connections compared with later developing neurons and 2) tend to establish more connections. Note, however, that the second effect might be limited, because, although few neurons are already occupied early in development, the number of neurons, and therefore potential partners, is lower as well. Our results concerning the relation between the filling fraction and the competition factor for establishing synapses could be tested experimentally for local neuronal connections.

Conclusion

Simple models that assume a straight outgrowth of axons in a randomly chosen direction can account for the exponential decay in the connection length distribution that is found experimentally in diverse neural networks in the primate and rat brain as well as the nematode C. elegans. In addition, spatial occupation of postsynaptic neurons and axonal competition for available synaptic targets leads to filling fractions that are similar to those found experimentally. The model is a simple starting point for modeling neuronal network development and may turn out to be incomplete when compared with anatomical data sets that can be characterized more extensively by a larger set of network parameters. In such a case, some of the factors that are now not part of our simple model would need to be included. We hope that this study will encourage both the modeling of neuronal network development as well as anatomical measurements of neuronal connectivity.

Supplementary Material

Supplementary Figures S1 and S2 can be found at: http://www.cercor.oxfordjournals.org/

Funding

Engineering and Physical Sciences Research Council (EP/E002331/1 and EP/G03950X/1 to M.K.) and the Royal Society (RG/2006/2 to M.K.).

We thank Yoonsuck Choe for providing us with spatial position data of Caenorhabditis elegans neurons, and Miles Whittington and Evelyne Sernagor for helpful comments on the manuscript. Conflict of interest: None declared.

References

Achacoso
TB
Yamamoto
WS
AY's neuroanatomy of C. elegans for computation
 , 
1992
Boca Raton (FL)
CRC Press
Amirikian
B
A phenomenological theory of spatially structured local synaptic connectivity
PLoS Comput Biol
 , 
2005
, vol. 
1
 pg. 
e11
 
Baruch
L
Itzkovitz
S
Golan-Mashiach
M
Shapiro
E
Segal
E
Using expression profiles of Caenorhabditis elegans neurons to identify genes that mediate synaptic connectivity
PLoS Comput Biol
 , 
2008
, vol. 
4
 pg. 
e1000120
 
Binzegger
T
Douglas
RJ
Martin
KAC
A quantitative map of the circuit of cat primary visual cortex
J Neurosci
 , 
2004
, vol. 
24
 (pg. 
8441
-
8453
)
Borisyuk
R
Cooke
T
Roberts
A
Stochasticity and functionality of neural systems: mathematical modelling of axon growth in the spinal cord of tadpole
Biosystems
 , 
2008
, vol. 
93
 (pg. 
101
-
114
)
Braitenberg
V
Schuez
A
Cortex: statistics and geometry of neuronal connectivity
 , 
1998
Heidelberg
Springer
Butz
M
Wörgötter
F
Van Ooyen
A
Activity-dependent structural plasticity
Brain Res Rev
 , 
2009
 
Forthcoming
Cabelli
RJ
Hohn
A
Shatz
CJ
Inhibition of ocular dominance column formation by infusion of Nt-4/5 or Bdnf
Science
 , 
1995
, vol. 
267
 (pg. 
1662
-
1666
)
Carmichael
ST
Price
JL
Architectonic subdivision of the orbital and medial prefrontal cortex in the macaque monkey
J Comp Neurol
 , 
1994
, vol. 
346
 (pg. 
366
-
402
)
Chklovskii
DB
Exact solution for the optimal neuronal layout problem
Neural Comput
 , 
2004
, vol. 
16
 (pg. 
2067
-
2078
)
Choe
Y
McCormcik
BH
Koh
W
Network connectivity analysis on the temporally augmented C. elegans web: a pilot study
Soc Neurosci Abstr
 , 
2004
, vol. 
30
 (pg. 
921
-
929
)
Costa
LdF
Kaiser
M
Hilgetag
CC
Predicting the connectivity of primate cortical networks from topological and spatial node properties
BMC Syst Biol
 , 
2007
, vol. 
1
 pg. 
16
 
Costa
LdF
Rodrigues
FA
Travieso
G
Villas Boas
PR
Characterization of complex networks: a survey of measurements
Adv Phys
 , 
2007
, vol. 
56
 (pg. 
167
-
242
)
Crepel
F
Regression of functional synapses in the immature mammalian cerebellum
Trends Neurosci
 , 
1982
, vol. 
5
 (pg. 
266
-
269
)
Easter
SS
Jr
Purves
D
Rakic
P
Spitzer
NC
The changing view of neural specificity
Science
 , 
1985
, vol. 
230
 (pg. 
507
-
511
)
Felleman
DJ
van Essen
DC
Distributed hierarchical processing in the primate cerebral cortex
Cereb Cortex
 , 
1991
, vol. 
1
 (pg. 
1
-
47
)
Hellwig
B
A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex
Biol Cybern
 , 
2000
, vol. 
82
 (pg. 
111
-
121
)
Hellwig
B
Schuz
A
Aertsen
A
Synapses on axon collaterals of pyramidal cells are spaced at random intervals: a Golgi study in the mouse cerebral cortex
Biol Cybern
 , 
1994
, vol. 
71
 (pg. 
1
-
12
)
Hilgetag
CC
Burns
GAPC
O'Neill
MA
Scannell
JW
Young
MP
Anatomical connectivity defines the organization of clusters of cortical areas in the macaque monkey and the cat
Phil Trans R Soc Lond B
 , 
2000
, vol. 
355
 (pg. 
91
-
110
)
Hume
RI
Purves
D
Geometry of neonatal neurones and the regulation of synapse elimination
Nature
 , 
1981
, vol. 
293
 (pg. 
469
-
471
)
Kaiser
M
Hilgetag
CC
Modelling the development of cortical networks
Neurocomputing
 , 
2004
, vol. 
58–60
 (pg. 
297
-
302
)
Kaiser
M
Hilgetag
CC
Nonoptimal component placement, but short processing paths, due to long-distance projections in neural systems
PLoS Comput Biol
 , 
2006
, vol. 
2
 pg. 
e95
 
Kaiser
M
Hilgetag
CC
Development of multi-cluster cortical networks by time windows for spatial growth
Neurocomputing
 , 
2007
, vol. 
70
 (pg. 
1829
-
1832
)
Kaiser
M
Martin
R
Andras
P
Young
MP
Simulation of robustness against lesions of cortical networks
Eur J Neurosci
 , 
2007
, vol. 
25
 (pg. 
3185
-
3192
)
Kalisman
N
Silberberg
G
Markram
H
The neocortical microcircuit as a tabula rasa
Proc Natl Acad Sci USA
 , 
2005
, vol. 
102
 (pg. 
880
-
885
)
Kaufman
A
Dror
G
Meilijson
I
Ruppin
E
Gene expression of C. elegans neurons carries information on their synaptic connectivity
PLoS Comput Biol
 , 
2006
, vol. 
2
 pg. 
e167
 
Kisvarday
ZF
Martin
KA
Freund
TF
Magloczky
Z
Whitteridge
D
Somogyi
P
Synaptic targets of HRP-filled layer III pyramidal cells in the cat striate cortex
Exp Brain Res
 , 
1986
, vol. 
64
 (pg. 
541
-
552
)
Kötter
R
Online retrieval, processing, and visualization of primate connectivity data from the CoCoMac database
Neuroinformatics
 , 
2004
, vol. 
2
 (pg. 
127
-
144
)
LaMantia
AS
Rakic
P
Axon overproduction and elimination in the corpus callosum of the developing rhesus monkey
J Neurosci
 , 
1990
, vol. 
10
 (pg. 
2156
-
2175
)
Lewis
JW
Essen
DCV
Architectonic parcellation of parieto-occipital cortex and interconnected cortical regions in the macaque monkey
J Comp Neurol
 , 
2000
, vol. 
428
 (pg. 
79
-
111
)
Lohmann
H
Rorig
B
Long-range horizontal connections between supragranular pyramidal cells in the extrastriate visual cortex of the rat
J Comp Neurol
 , 
1994
, vol. 
344
 (pg. 
543
-
558
)
Lohof
AM
DelhayeBouchaud
N
Mariani
J
Synapse elimination in the central nervous system: functional significance and cellular mechanisms
Rev Neurosci
 , 
1996
, vol. 
7
 (pg. 
85
-
101
)
McGuire
BA
Gilbert
CD
Rivlin
PK
Wiesel
TN
Targets of horizontal connections in macaque primary visual cortex
J Comp Neurol
 , 
1991
, vol. 
305
 (pg. 
370
-
392
)
Murray
JD
Mathematical biology
 , 
1990
Heidelberg
Springer
Nisbach
F
Kaiser
M
Time domains for spatial network development generate multiple-cluster small-world networks
Eur Phys J B
 , 
2007
, vol. 
58
 (pg. 
185
-
191
)
Purves
D
Lichtman
JW
Elimination of synapses in the developing nervous system
Science
 , 
1980
, vol. 
210
 (pg. 
153
-
157
)
Rakic
P
Development of visual centers in the primate brain depends on binocular competition before birth
Science
 , 
1981
, vol. 
214
 (pg. 
928
-
931
)
Rakic
P
Mechanism of ocular dominance segregation in the lateral geniculate nucleus: competitive elimination hypothesis
Trends Neurosci
 , 
1986
, vol. 
9
 (pg. 
11
-
15
)
Rakic
P
Neurogenesis in adult primate neocortex: an evaluation of the evidence
Nat Rev Neurosci
 , 
2002
, vol. 
3
 (pg. 
65
-
71
)
Rakic
P
Bourgeois
JP
Eckenhoff
MF
Zecevic
N
Goldman-Rakic
PS
Concurrent overproduction of synapses in diverse regions of the primate cerebral cortex
Science
 , 
1986
, vol. 
232
 (pg. 
232
-
235
)
Rakic
P
Riley
KP
Overproduction and elimination of retinal axons in the fetal rhesus monkey
Science
 , 
1983
, vol. 
219
 (pg. 
1441
-
1444
)
Samsonovich
AV
Ascoli
GA
Statistical morphological analysis of hippocampal principal neurons indicates cell-specific repulsion of dendrites from their own cell
J Neurosci Res
 , 
2003
, vol. 
71
 (pg. 
173
-
187
)
Sanes
JR
Lichtman
JW
Development of the vertebrate neuromuscular junction
Annu Rev Neurosci
 , 
1999
, vol. 
22
 (pg. 
389
-
442
)
Sperry
RW
Chemoaffinity in the orderly growth of nerve fiber pattern and connections
Proc Natl Acad Sci USA
 , 
1963
, vol. 
50
 (pg. 
703
-
710
)
Sporns
O
Chialvo
DR
Kaiser
M
Hilgetag
CC
Organization, development and function of complex brain networks
Trends Cogn Sci
 , 
2004
, vol. 
8
 (pg. 
418
-
425
)
Stepanyants
A
Hof
PR
Chklovskii
DB
Geometry and structural plasticity of synaptic connectivity
Neuron
 , 
2002
, vol. 
34
 (pg. 
275
-
288
)
Stephan
KE
Kamper
L
Bozkurt
A
Burns
GA
Young
MP
Kotter
R
Advanced database methodology for the Collation of connectivity data on the macaque brain (CoCoMac)
Philos Trans R Soc Lond B Biol Sci
 , 
2001
, vol. 
356
 (pg. 
1159
-
1186
)
Stephan
KE
Zilles
K
Kotter
R
Coordinate-independent mapping of structural and functional data by objective relational transformation (ORT)
Philos Trans R Soc Lond B Biol Sci
 , 
2000
, vol. 
355
 (pg. 
37
-
54
)
Tarui
T
Takahashi
T
Nowakowski
RS
Hayes
NL
Bhide
PG
Caviness
VS
Overexpression of p27(Kip1), probability of cell cycle exit, and laminar destination of neocortical neurons
Cereb Cortex
 , 
2005
, vol. 
15
 (pg. 
1343
-
1355
)
Uylings
HB
van Pelt
J
Measures for quantifying dendritic arborizations
Network
 , 
2002
, vol. 
13
 (pg. 
397
-
414
)
van Ooyen
A
Competition in the development of nerve connections: a review of models
Network
 , 
2001
, vol. 
12
 (pg. 
R1
-
R47
)
van Ooyen
A
Modeling neural development
 , 
2003
Cambridge, MA
MIT Press
van Ooyen A
Competition in neurite outgrowth and the development of nerve connections
Prog Brain Res
 , 
2005
, vol. 
147
 (pg. 
81
-
99
)
van Ooyen
A
Van Pelt
J
Corner
MA
Implications of activity dependent neurite outgrowth for neuronal morphology and network development
J Theor Biol
 , 
1995
, vol. 
172
 (pg. 
63
-
82
)
van Ooyen
A
Willshaw
DJ
Competition for neurotrophic factor in the development of nerve connections
Proc Biol Sci
 , 
1999
, vol. 
266
 (pg. 
883
-
892
)
van Ooyen
A
Willshaw
DJ
Ramakers
GJA
Influence of dendritic morphology on axonal competition
Neurocomputing
 , 
2000
, vol. 
32
 (pg. 
255
-
260
)
Verhage
M
Maia
AS
Plomp
JJ
Brussaard
AB
Heeroma
JH
Vermeer
H
Toonen
RF
Hammer
RE
van den Berg
TK
Missler
M
, et al.  . 
Synaptic assembly of the brain in the absence of neurotransmitter secretion
Science
 , 
2000
, vol. 
287
 (pg. 
864
-
869
)
Watts
DJ
Strogatz
SH
Collective dynamics of ‘small-world’ networks
Nature
 , 
1998
, vol. 
393
 (pg. 
440
-
442
)
Wen
Q
Chklovskii
DB
A cost-benefit analysis of neuronal morphology
J Neurophysiol
 , 
2008
, vol. 
99
 (pg. 
2320
-
2328
)
White
JG
Southgate
E
Thomson
JN
Brenner
S
The structure of the nervous system of the nematode C. elegans
Phil Trans R Soc of London B
 , 
1986
, vol. 
314
 pg. 
1
 
Wiesel
TN
Postnatal-development of the visual-cortex and the influence of environment
Nature
 , 
1982
, vol. 
299
 (pg. 
583
-
591
)
Yamamoto
N
Tamada
A
Murakami
F
Wiring of the brain by a range of guidance cues
Prog Neurobiol
 , 
2002
, vol. 
68
 (pg. 
393
-
407
)