Morphological Diversity Strongly Constrains Synaptic Connectivity and Plasticity

Synaptic connectivity between neurons is naturally constrained by the anatomical overlap of neuronal arbors, the space on the axon available for synapses, and by physiological mechanisms that form synapses at a subset of potential synapse locations. What is not known is how these constraints impact emergent connectivity in a circuit with diverse morphologies. We investigated the role of morphological diversity within and across neuronal types on emergent connectivity in a model of neocortical microcircuitry. We found that the average overlap between the dendritic and axonal arbors of different types of neurons determines neuron-type speci ﬁ c patterns of distance-dependent connectivity, severely constraining the space of possible connectomes. However, higher order connectivity motifs depend on the diverse branching patterns of individual arbors of neurons belonging to the same type. Morphological diversity across neuronal types, therefore, imposes a speci ﬁ c structure on ﬁ rst order connectivity, and morphological diversity within neuronal types imposes a higher order structure of connectivity. We estimate that the morphological constraints resulting from diversity within and across neuron types together lead to a 10-fold reduction of the entropy of possible connectivity con ﬁ gurations, revealing an upper bound on the space explored by structural plasticity.


Introduction
The connectome-the comprehensive map of neural connectionsis believed to determine the functional capabilities of the brain (Sporns et al. 2000;Honey et al. 2009). The connectome is not fixed but changes continuously through structural plasticity mechanisms (Feldman 2009;Hofer et al. 2009;Holtmaat and Svoboda 2009), making it a primary substrate for learning and memory. Storage capacity is limited by the number of configurations of connectivity that can be reached through plasticity (Chklovskii et al. 2004;Lamprecht and LeDoux 2004). Assuming 10% connectivity and no other constraints, even a comparatively small, purely local microcircuit of 10 000 neurons would have roughly × 2.86 10 14116758 theoretically possible network configurations (Fig. 1A, red). In reality, however, rewiring through plasticity is subject to a number of known constraints and probably many unknown constraints as well. This makes it difficult to estimate the number of "biologically available" configurations. What we can do, is use the shape and extent of the dendritic and axonal arborizations (Fig. 1B,C) and the number of synapses required to form a viable connection (Fig. 1D) to define the set of "anatomically constrained" configurations ( Fig. 1E)-a superset of the biologically available configurations. The size of this set, as an approximation of the set of biologically available configurations, places an upper bound on the memory capacity of the microcircuit. While anatomical constraints may be known in principle, we do not know the anatomical level of detail at which they operate. In particular, it is unclear whether individual neuron types can be represented by their "average morphology," that is, the probability distribution of dendrite and axon segments around the soma , or whether connectivity also depends on morphological variability within neuron types (Fig. 1F).
Current experimental approaches cannot adequately address these questions. For example, tract tracing and diffusion tensor imaging have provided maps of the connections between brain areas and regions on the scale of the whole mammalian brain, but do not resolve individual neurons and synapses (Kuan et al. 2014). Whole-cell paired recordings have advanced the detailed characterization of the anatomy and physiology of synaptic connections, in particular 3D reconstructions of axons, dendrites, and their putative points of contact (potential synapse locations). However, this has only been possible for a miniscule number of connection types per species (Markram et al. 1997;Feldmeyer et al. 2002;Wang et al. 2002;Le Bé et al. 2007; Silberberg and Markram 2007).
Experimental studies using electron microscopy (EM) have yielded a complete map of connectivity between the 302 neurons in the roundworm, Caenorhabditis elegans (White et al. 1986;Varshney et al. 2011), and similar methods have been applied to mammalian nervous systems (Mishchenko et al. 2010;Kasthuri et al. 2015). However, the volumes of neural tissue that have been reconstructed in mammals are smaller than the size of a single dendrite and thus too small to determine all the local connections in a microcircuit, or even all the afferent connections of only a single neuron. Furthermore, even scaledup EM reconstructions would provide just a few out of a vast number of biologically available configurations. Such reconstruction would not address the question of how many and which configurations are reachable by structural plasticity.
To evaluate anatomically constrained connectivity, we analyzed a model of neocortical tissue from the somatosensory cortex of juvenile rat . This model implements 3 anatomical constraints on connectivity (synapses a subset of axo-dendritic appositions; number of synapses limited by axonal length; connections formed by multiple synapses; see also below, Hill et al. 2012;Reimann et al. 2015). We  began by studying the impact of the constraints on the connections formed by a single neuron. We then performed detailed, analyses of the connection matrices of the modeled microcircuits, each of which represents a random instance of anatomically constrained connectivity within a cylindrical volume with a diameter of~440 μm and a thickness of~2082 μm. The analyses characterized the connectivity structure of the model with respect to a number of nonrandom features of biological connectivity, previously characterized in in vitro experiments. We found that most of the biologically characterized structure emerges when constraints based on the "average" morphologies of different neuron types are applied. However, higher order connectivity motifs, such as clustered connectivity depend on individual variability within neuron types (Perin et al. 2011). Since the connection matrices were not subject to rewiring through plasticity, we conclude that the initial emergence of first and higher order structure does not require experience-driven synaptic plasticity. On this basis, we hypothesize that the upper bound on the size of the set of biologically available configurations, provided by anatomical constraints alone, provides a useful approximation of its limits.

Entropy of Constrained Connectivity
We analyzed the entropy of the efferent connectivity of individual PCs in layer 5 of the model described in . For a given PC and each other neuron in this model, we considered the probability distribution of the number of synapses connecting the pair, ( = ) The entropy was then calculated as: where → n i j is the number of synapses formed by neuron i on neuron j. Assuming only a single constraint of a maximum of 20 synapses between pairs of neurons (a biologically relevant range) entropy is maximized for the uniform distribution on the interval between 0 and 20. Further constraints could then be applied by changing the ( = ) → P n x i j for connections to individual neurons or neuron types. Either by counting the number of axo-dendritic appositions, that is, potential synaptic locations, between the pair and setting it ( = )= → P n x 0 i j for all x above that maximum value; and/or by prescribing a probability distribution for the number of synapses per connections to individual neuron types, based on experimental data or predicted from the average number of appositions between pairs as described in . Additionally, a probability distribution for the total number of synapses formed by the PC onto all other neurons could be applied. For the handling of this case see Supplementary material.

Analysis of Instances of Constrained Connectivity
We analyzed the connectivity of 7 instances of modeled microcircuitry described in . We extracted the connectivity between the approximately 31 000 neurons of each instance in the form of a connection matrix M, detailing the presence or absence of a connection between all pairs of neurons. The matrix was further subdivided into 3025 submatrices of connectivity between 55 individual neuron types:

Common Neighbor Bias
The matrix of the number of common neighbors between types m i and m j was calculated as: The result has the shape × N N i j . With CN i j , and → M m m i j , we calculate the global clustering coefficient for connections from m i to m j as follows. The sum over all entries in CN i j , yields the number of triangular motifs with an m i neuron in one corner, an m j neuron in another corner and a neuron in layer L in the third. The sum over all entries where the corresponding entry in → M m m i j is 1 (i.e., a connection exists) yields the number of closed triangles. The ratio of closed to all triangles is then the global clustering coefficient for connections from m i to m j .

Control Connectivity Preserving Individual Distance Dependence
The number of common neighbors and clustering coefficients were compared to results from randomized connection matrices that preserved the distance dependence of all connection types, generated in the following way: For all connection types , that is, connectivity within a neuron type, the entries along the main diagonal were always kept 0 (no autapses).
The number of distance bins d was determined as follows: Starting with = d 1, we generated 10 randomized connection matrices for → m m i j . We then calculated for the original and randomized matrices the probability distribution of soma distances of "connected" pairs of neurons. If a Kolmogorov-Smirnov (KS) test of the difference between the distributions yielded no significance difference ( > P 0.05) for all 10 randomized matrices, we concluded that the distance dependence was preseved, otherwise the procedure was repeated with an incremented value of d.
We assembled randomized connection matrices with preserved distance dependence for the whole microcircuit by repeating this procedure for all 55 × 55 = 3025 possible pairs of neuron types.

Recreating a Connectivity Sampling Experiment
We compared connectivity in anatomically constrained connectivity to results from (Perin et al., 2011) by recreating their in vitro connectivity sampling experiment in silico. We obtained the relative coordinates of the patch pipettes used in 43 of their experiments (8.7 ± 2.7 coordinates per experiment; mean pairwise distance: 105 μm) inside layer 5 of the modeled microcircuitry with random offsets and picked for each resulting location the closest PC. Connectivity was then investigated for all × ( − ) N N 1 2 pairs in a set. Each set of coordinates was randomly placed 10 times, leading to 430 analyzed sets with a total of 16 020 pairs of neurons.

Calculation of Average Dendrite and Axon Clouds
Average morphologies were calculated by considering 10 instances of each available reconstructed morphology of a neuron type, assigning it a random rotation around the vertical axis and aligning them all at the soma. We projected the result along the depth axis and then calculated for each 2×2 μm pixel the mean dendrite or axon density (μm of axon/dendrite length per pixel). We call the results V m

Using Average Clouds to Connect Neurons
To generate connections from neuron type m i to m j , i.e., to generate → M m m cloud i j we first calculated the "overlap" of the types at various spatial offsets as the convolution of their axon and dendrite clouds,  Each entry in → P m m cloud i j was then considered the probability to find a connection in the indicated direction between the corresponding pair of neurons. Probabilities for different pairs were otherwise statistically independent. The matrix of connection probabilities could then be used to generate any number of random instances of cloud-based connectivity from m i to m j .
Repeating this procedure for all 55 × 55 = 3025 possible pairs of neuron types we assembled connection matrices for the whole microcircuit.

Results
The biologically driven in silico modeling effort described in Markram et al. (2015) uses 55 morphological neuron types, resulting in 3025 theoretically possible "connection types" (sets of pairwise connections between neurons of specific morphological types, for example, pyramidal cells (PCs) in layer 4 to large basket cells (LBCs) in layer 4; see also Supplementary Table S1). The algorithm used to generate the local connectivity applied the following 3 morphological and anatomical principles of biological connectivity, parameterized with biological data, as constraints that drastically reduced the number of connections in the model: (1) The number of synapses forming a connection is close to a number larger than 1 and predictable from the average strength of axo-dendritic overlap of the connection type (Fig. 1D); (2) the total number of efferent synapses on a neuron is limited by the space available on the axon (Fig. 1B); (3) synaptic locations are a subset of axo-dendritic appositions, that is, the connections formed by a neuron are limited to the regions reached by its axon (Fig. 1C), which are determined by its morphology and the laminar structure of the neocortex.
To evaluate the degree to which these constraints limit the number of available configurations, we calculated the mean entropy of the connections formed by individual PCs in layer 5 before and after application of the constraints (see Materials and Methods section). This entropy can be thought of as a measure of the uncertainty in the presence and number of synapses formed by neuron i onto any other neuron j.
n i j is the number of synapses formed by neuron i on neuron j and n max is the highest number of synapses it can form.
Without these constraints, each neuron is potentially capable of innervating any of N = 31 000 local neurons with any realistic number of synapses, which we set to be between 0 and = n 20 max ( Fig. 2A, top). According to the principle of maximum entropy, we set ( . This yields approximately 145 000 bits of entropy. In the anatomically constrained case (Fig. 2B, middle), entropy is reduced by prescribing narrower, nonuniform distributions, ( = ). We found that applying any 1 of these constraints led to a reduction in entropy of approximately 40% to approximately 86 000 bits. Applying constraint 2 combined with constraint 3 led to an additional reduction of just 22%, indicating that the 2 constraints are partially redundant. Conversely, the 2 other possible combinations (1 and 2 or 1 and 3) produced reductions close to 80%, indicating very little redundancy. Finally, when all 3 constraints were combined, entropy was reduced by about 90%. Considering that, in this context, the entropy is proportional to the binary logarithm of the number of available wiring configurations, this corresponds to a~10 39000 fold reduction in the number of anatomically feasible configurations.
It is conceivable that other topological constraints that have been observed in experiments further reduce the space of possible configurations. These include distance-dependent connectivity (Holmgren et al. 2003;Perin et al. 2011), the nonrandom structure of interlaminar connectivity (Bannister 2005;Shepherd et al. 2005;Helmstaedter et al. 2008), nonrandom "targeting" of individual neuron types (Callaway 2002;Watts and Thomson 2005;Yoshimura and Callaway 2005;Wozny and Williams 2011), and clustering of connectivity (Song et al. 2005;Perin et al. 2011). However, it is also possible that these additional constraints are partially or completely redundant once anatomical constraints have been taken into account.
To explore the extent to which these biological features constrain connectivity, we performed a comprehensive analysis of the Morphological Diversity Strongly Constrains Synaptic Connectivity and Plasticity Reimann et al. | 4573 connectivity generated by just applying the anatomical constraints. We, therefore, analyzed the connection matrices for 7 instances of a model of neocortical microcircuitry as described in Reimann et al. 2015, from here on: Neocortical Microcircuit or "NMC model"), that is, 7 randomly chosen instances of anatomically constrained connectivity. Each matrix summarizes the connectivity between~31 000 neurons, belonging to 55 layerspecific neuron types, connected via~36.5 million synapses in~7.5 million multi-synaptic connections. The analysis was applied to NMC connectivity for all possible pairs of neuron types, allowing comparisons against the small subset whose connectivity has been measured experimentally, but also predictions for all other types.

Distance-Dependent Connection Probabilities
Local neuronal connectivity at the resolution of single neurons and connections is commonly assessed in slice preparations at distances up to about 100 μm, occasionally up to 150 μm (Holmgren et al. 2003;Le Bé et al. 2007;Silberberg and Markram 2007). Such studies have estimated excitatory connection probabilities in cortex in the range 0.03-0.15. An equivalent analysis of the NMC connectivity for pairs of neurons lying within 100 μm of each other, yielded a mean connection probability of 0.065 ± 0.001. The main anatomical constraint underlying this value was the space available on the axon to form connections: any higher value would have required more densely packed synapses or longer axons.
Previous studies have shown that the connectivity is not uniform, but strongly distance dependent (Holmgren et al. 2003;Perin et al. 2011). To investigate this, we characterized the distance-dependent connection probabilities (C P ) for individual connection types in the 7 connection matrices (see Fig. 3A). We found that, as in biology, C P s fell as the distance between neurons grew and their local arborizations separated. However, we also found that the "rate" at which they fell was connection-type specific. For example, C P s for intralaminar connections from Martinotti cells (MCs) or large Basket cells (BCs) to PCs fall more rapidly than those for connections from PCs to PCs: at short distances, C P s for MC to PC connections are higher than for PC to PC connections while at distances beyond 200 μm this relationship is inverted (Fig. 3B). Since the number of potential synaptic partners grows linearly with lateral distance, relatively small differences in C P for higher distances translate into large difference in the total number of connected neurons (see Supplementary Fig. S1).
For a more systematic analysis, we performed exponential fits of the distance-dependent fall-off of C P for all connection types and calculated their respective space constants (Fig. 3C). These proved to be highest for connections between excitatory neurons and lowest for connections between inhibitory neurons (see Supplementary Fig. S2). The predicted neuron typespecific distance dependencies are a strong indicator of nonrandom structure in the connectivity and, therefore, further evidence of the strength of anatomical constraints.
Comparing connection probability profiles to literature is somewhat problematic, as the various results were often obtained from different animals, brain regions and ages and consequently vary drastically (Fig. 3B). Also, connectivity analyzed in vitro is potentially subject to sampling biases of distances or directions relative to the layer boundaries. Finally, both in vitro experiments and the NMC model are likely to underestimate connectivity due to slicing artifacts, which have been shown to be substantial (Stepanyants et al. 2009). We expect the effect to be stronger in the NMC model, although it employs a limited repair process which attempts to compensate for cut axons in morphological reconstructions from cortical slices (Markram  . This is because in vitro the non-sliced portions of axons and dendrites are still aligned in the same plane, allowing them to meet, whereas the model assigns a random rotation to neurons. Slicing affects upwards connections more strongly (Stepanyants et al. 2009), and indeed the NMC model matched the literature well for downward connections from layers 2/3 and 4-5 and for intralaminar connections within layers 2/3 and 5, but generally underestimated upward connection probabilities (Fig. 3B, see also Supplementary Figs S3-S5). The model did not match the dense inhibitory connectivity within a layer reported from mouse cortex (Fino and Yuste 2011;Jiang et al. 2015), but corresponding values reported for rat are significantly lower (Thomson et al. 2002;Yoshimura and Callaway 2005). Finally, slicing is known to affect more distant connections generally more strongly (Stepanyants et al. 2009), thus the true space constants of distance-dependent connectivity are likely to be larger than in the NMC model.

Large-Scale Patterns of Interlaminar Connectivity
In the NMC model, the laminar structure and diversity of the neocortex was implemented through layer-specific neuronal densities and compositions (Fig. 4A) as well as layer-specific reconstructed morphologies for each neuron type. We expected that this anatomical specificity would impose large-scale structure onto synaptic connectivity within and across layers, which we characterized in terms of synapse numbers in connections within and across layers, both in total and normalized by neuron count (see Supplementary material). We also compared these numbers to results from the literature, albeit from a different animal and brain region (Binzegger et al. 2004, see Materials and Methods section).
Layer 1 was characterized by low neuron densities and thus contributed only a tiny proportion of all synapses (Fig. 4A, B). Synapse numbers in the other layers increased with cortical Presynaptic  Blue: from MCs, red: from BCs; green: from other PCs. Circles and error bars indicate mean and SD of sampled probabilities (N = 7 instances of connectivity), lines represent exponential fits to the data. Dashed lines indicate results from the literature. Colors as above. X-marks: (Lefort et al. 2009); stars: (Jiang et al. 2015); triangles: 2/ 3 to 2/3: (Thomson et al. 2002;Holmgren et al. 2003;Yoshimura and Callaway 2005); 5 to 5: (Thomson et al. 2002;Perin et al. 2011); 4 to 2/3: (Feldmeyer et al. 2002). An upwards pointing arrow with labeled connection probability is used where it lies beyond the limits of the y-axis. (C) Space constants of the exponential fits for all connection types where the fit was possible (at least 1 pair of neurons at a lateral distance ≤25 μm). Excitatory morphological types are surrounded by red lines.
Morphological Diversity Strongly Constrains Synaptic Connectivity and Plasticity Reimann et al. | 4575 depth (82% increase from L2/3 to L6) but were all within the same order of magnitude. The ratio of interlaminar output to interlaminar input was highest in layer 1 (13.6 ± 1.0, mean ± std), decreasing sharply through layers 2/3 (5.82 ± 0.08), 4 (1.64 ± 0.02), 5 (0.45 ± 0.01), and 6 (0.48 ± 0.01). These findings reveal a highly polar structure with a net flow of connections from more superficial to deeper layers. Similarly, the data of Binzegger et al. (2004) reveal a reduction in the ratio of out-to inputs from 1.71 in L2/3 to 0.67 in L5, but followed by an increase to 2.89 in L6. The reason for this discrepancy is possibly that the model is based on data from juvenile rats, where ascending axons are not yet fully mature, or the slicing artifact discussed above. Overall, numbers of intralaminar and interlaminar synapses were approximately equal (18.3 M vs. 17.7 M, respectively) as they were in Binzegger et al. (2004) (intralaminar divided by interlaminar synapse count: 0.99). However, there were strong differences between layers. Neurons in layer 1 received most of their synapses from neurons in other layers (intralaminar/ interlaminar quotient: 0.42 ± 0.04, mean ± std), while neurons in layers 2/3 received 2.73 ± 0.06 times more synapses from neurons in the same layer than from neurons in other layers. In layers 4 and 5, the ratio was much lower (0.57 ± 0.01 and 0.62 ± 0.01 respectively), but rose again in layer 6 (1.77 ± 0.02). This overall trend was comparable to the results of Binzegger et al. (2004), although they found even less intralaminar input into layers 5 and 6 (ratio for 2/3:2.96, 4:0.7, 5:0.17, 6:0.69).
Although these results only refer to connections originating and terminating within the modeled microcircuitry, they confirm the nonrandom structure of its interlaminar connectivity. Further investigation showed that neurons in layers 1 and 4 receive relatively few intralaminar connections and that they both provide more trans-laminar connectivity to other layers than they receive (see above). This is consistent with their role as primary targets for white matter projections (Felleman and Van Essen 1991).
Analysis of the mean number of synapses formed by individual neurons in the different layers (Fig. 4C) further characterized layers 1, 2/3, and 4 as sources, and layers 5 and 6 as recipients of interlaminar connections with strong recurrent connectivity. It also revealed that the apparent weakness of connections from layer 1 was mainly due to low-neuron density. In fact, individual layer 1 neurons were one of the strongest interlaminar inhibitors.
The ratio of excitatory to inhibitory afferent synapses formed a gradient, increasing monotonically with cortical depth, from 0.16 ± 0.01 in layer 1 to 4.56 ± 0.08 in layer 6. It thus appears that excitatory connections have a stronger downwards bias than inhibitory connections. Indeed, 63 ± 0.2% of excitatory connections from layers 2/3 and 4 targeted layers 5 or 6; the equivalent proportion for inhibitory connections was only 43% ± 0.5%. These numbers are lower than estimated ratios of excitatory to inhibitory synapses obtained from EM data (Kasthuri et al. 2015) and the predictions of Binzegger et al. (2004) (from 5.3 for neurons in L2/3 monotonically increasing to 16.4 for neurons in L6). Since connectivity in our model is limited to local intracortical connections, and external and extracortical input is largely excitatory, this discrepancy is expected. The strong increase of excitatory to inhibitory balance with cortical depth, however, matches the result of (Binzegger et al. 2004) closely. Assuming the missing connections are accounted for by long range intracortical, white matter cortical and extracortical projections, we can estimate the contributions of the latter to the synaptic anatomy of the microcircuit to be 80 ± 2% of all synapses .
Taken together, these results confirm that anatomical constraints impose a specific structure on connectivity, reflected in aggregate synapse counts for individual layers. While this highlevel structure is not surprising, this is the first time it has been characterized extensively. We continued our analysis by characterizing the specificity of connectivity in the NMC model on a more fine-grained level, taking individual neuron types within a layer into account.

Targeting Morphological Types
Structure in the connectivity of neural microcircuitry extends beyond laminarity, with different neuron types within a layer preferentially sending synaptic output to and receiving input from a subset of other neuron types in the same and other layers Helmstaedter et al. 2008;Brown and Hestrin 2009). This pattern is frequently explained in terms of specific molecular targeting mechanisms and experience-driven plasticity (Somogyi 1977;Somogyi et al. 1998;Hofer et al. 2009;Holtmaat and Svoboda 2009). To examine the role of purely anatomical constraints in the absence of such mechanisms, we compared the NMC connectivity to statistical control cases with respect to the degree of specificity in the neuron types innervated by the axons of individual neurons. While the interlaminar structure examined above may result largely from the laminar placement of neurons, any difference in the targeting of neuron types within the same layer must be a consequence of their morphological differences.
For the 7 instances of connectivity, we generated 2 "neuron type innervation profiles"-representing for each neuron the presence or absence of inputs/outputs from/to each of the 55 neuron types. The input profile was represented by a 55 bit vector per neuron [ ( )] I n in , where the value of ( ) I n in was 1 if neuron n received at least 1 input from a neuron belonging to type j, otherwise 0. The output profile, [ ( )] I n out , was calculated in the same way, but for neuron outputs (for examples see Fig. 5A).
If-hypothetically-connectivity were completely unstructured, according to the principle of maximum entropy, innervation of each neuron type would be statistically independent of all other types with a probability of 0.5. Consequently, all of the 2 55 possible profiles would be equally likely and the possibility of finding 2 duplicate profiles would be vanishingly small. We, therefore, tested each profile for the presence of duplicates. The results showed that 27% of the output profiles and 9% of the input profiles were duplicates of existing profiles (Fig. 5B, red bars). As this result is virtually impossible in unstructured connectivity, we can conclude that the repeated patterns are not mere coincidence but an indication of a targeting of certain connection types.
The presence of duplicate profiles is an indication that some profiles are vastly more likely than others. To explore this question further, we analyzed the probability distribution of neuron type innervation profiles for individual neurons. Since each NMC model instance provides no more than~31 000 out of 0.00 0.02 0.04 0.06 0.8 0.1 0.12 0.14 0.16 0.18 0.20

Mutual information (bits)
Output patterns Input patterns Morphological Diversity Strongly Constrains Synaptic Connectivity and Plasticity Reimann et al. | 4577 more than 10 16 possible innervation profiles (1 data point per neuron), we also included the data from 35 additional instances, where the anatomical and morphological constraints on the connectivity were parameterized with data from 5 individual rats . In this way, we were able to sample 42 instances providing a total of 1 302 000 data points. We started by calculating the entropy of the distribution of profiles, which we estimated at 15 bits per neuron for the output profiles and 17 bits per neuron for the input profiles (Fig. 5B). In contrast, a completely unstructured circuit would have an actual entropy of 55 bits per neuron and, when undersampled with 1 302 000 data points, would yield an estimated entropy of ( ) ≅ log 1 302 000 20.3 2 bits per neuron. The difference is evidence that neurons are strongly constrained with respect to the neuron types they connect to.
To understand how this reduction in entropy relates to neuron morphology and its variability across and within neuron types, we generated 4 statistical control models, P, M, M × L, and M × M (see below, methods), each incorporating progressively more anatomical constraints than the previous one. For each control, we generated 1 instance based on each of our 42 NMC model instances, thus subjecting each to the same under-sampling as in the original analysis. Specifically, the first statistical control (P) was based on the "average" neuron type connection probability observed in the 42 NMC instances. , where m indicates a neuron type and | | m is the number of neurons of the type.
The P control is unstructured: the probability that the profiled neuron forms a connection with a neuron type is equal for all neurons and types. The M control accounts for differences in the number of neurons innervated by a given neuron type, but not for specific preferences-that is, it takes account of the overall size of dendritic/axonal trees, but not of their shape. Conversely, the M × L and M × M controls account for neuron type-specific preferences in the targeting of layers and/or neuron types, but not for individual neuron morphologies.
Analysis of connectivity in the 4 models showed higher redundancy and lower entropy for outputs than for inputs-a trend which became stronger as new constraints were added to the models. However, the differences between the M × L and M × M controls were negligible and even the M × M controls still displayed higher entropy than the NMC model (4.1 bits higher for outputs; 2.5 bits higher for inputs; Fig. 5C, blue lines).
The entropy of the P and the M controls was not maximal, because the probabilities used to construct the profiles were significantly different from 0.5-the value that yields maximal entropy (and the minimum number of duplicates). However, entropy in M × L and M × M was further reduced by correlations between the innervation of different neuron types. For example, the tendency of neurons to innervate neurons of type A may correlate with innervation of a second type B.
To quantify these correlations and their dependence on individual neuron variability, we calculated the mutual information (MI) between pairs of neuron types in the innervation profiles, that is, between individual rows of the profiles (〈 ( ) ( ) … ( ) 〉  I  I  I  n  1 , 2 , , j j j out out out , similarly for I in ) and compared the results for M × M with those for the NMC model (Fig. 5D). In P and M, MI was zero by definition. Results for M × L were comparable to those for M × M. The M × M output profiles were characterized by the presence of significant MI, mainly between specific neuron types in layers 2/3 and 4; these types also shared some MI with PCs in layer 5 and BCs and MCs in layer 6. However, in most cases, MI in the control model was far lower than in the NMC model. This was most pronounced for pairs of neuron types in the same layer, but also for combinations of pairs across layers 2/3, 4, and 5, where MI was more than 2-fold higher in the NMC model. Notably, only the NMC model displayed significant MI within layer 5. Neither the NMC model nor the M × M controls showed any significant MI between neuron types in layer 5 and types in layer 6, nor between types in layer 1 and any other neuron type.
In a similar analysis of input profiles, most of the MI in M × M was between neuron types in layer 6 and types in layers 2/3, 5, or 6. In the NMC model by contrast, we found additional MI between pairs in layers 4 and 5 and within layer 2/3. MI for pairs where 1 neuron type was in layer 6 was not further increased relative to M × M.
MI between pairs of neuron types can be interpreted as probabilistic connectivity rules, where the probability that a neuron innervates 1 of the neuron types in the pair is dependent on whether it innervates the other. For example, we predict that when a particular neuron innervates PCs in layer 2/3, the probability that the same neuron will innervate Inverted PCs (IPCs) in layer 6 is "reduced" (see Supplementary Fig. S6), but the probability that it innervates Star PCs (SPs) is "increased". Similarly for inputs: innervation of a neuron by an MC in layer 2/3 reduces the probability that it will be innervated by a Nest BC (NBC) in layer 6, but increases the probability that it will be innervated by an MC in L5.
These findings support the notion that anatomical constraints alone are sufficient to produce highly structured patterns of interlayer innervation and reveal a degree of specificity that goes beyond simple laminar targeting. Notably, however, this specificity only appears when we take account of morphological variability "within" neuron types.

Emergent Neuronal Assemblies
Previous studies have also identified higher-order connectivity motifs in biological microcircuits. For example, a study of connectivity in networks of PCs in layer 5 using simultaneous patch clamping of up to 12 cells showed that connected pairs of neurons in the network had more "common neighbors," that is, neurons connected to both neurons of the pair, than would be the case in a random network (Song et al. 2005;Perin et al. 2011). This suggested the presence of neuronal assembliesclusters of tightly-interconnected neurons.
To investigate whether anatomical constraints alone could give rise to such structures, we recreated the experiment reported in Perin et al. (2011) for 7 instances of NMC connectivity. As in the original experiment, we randomly selected regions of the model, and measured the connections of PCs in layer 5 at relative positions approximating those of patched neurons in the original experiment (see Materials and Methods section). We found that the number of connected pairs with more than 2 common neighbors was more than 10 times the value expected for a random network with the same overall connection probability (see Fig. 6A, gray bars vs. black line).
This result closely matched the results of (Perin et al. 2011) (Fig. 6A, green line).
While experiments are usually restricted to testing the common neighbor rule for 1 type of neuron at a time, we tested the rule for all possible pairs of neuron types in the NMC model. We first calculated the distribution of common neighbor numbers over all pairs of neurons belonging to the selected types, considering common neighbors of "any" type in the same layer (Fig. 6B). We then compared against the results for random control networks. To a certain degree, a distance-dependent connection probability alone can explain an increase in common neighbor numbers compared to an Erdős-Rényi model, as neurons in close proximity will have an equally elevated connection probability to neurons in their shared spatial neighborhood and, therefore, a large number of common neighbors. We, therefore, compared against controls with the same distance-dependent connection probabilities: for each instance of anatomically constrained connectivity, we generated 10 random instances with the same neuron positions and the same distance-dependent connection probability profiles for all connection types (see Materials and Methods section; as these controls also recreated the distance dependence of individual connection types (see Fig. 4), they preserved even more biological detail than the M × M model). We then performed the same common neighbor analysis for matching connection types in the controls. Finally, we used the mean difference between NMC and controls in the slope of log-linear fits to the distribution of common neighbors as a measure of the "strength of the common neighbor rule" for specific connection types (Fig. 6C).
We found that increased numbers of common neighbors numbers was a near universal feature of anatomically constrained connectivity (Fig. 6D). For all but 6 pairs of neuron types, the slope of common neighbor numbers was significantly steeper in the NMC model than the control networks (P < 0.01, one-tailed t-test, 7 instances against 7 × 10 = 70 random networks). Most of the connection types where the difference did not reach significance were in layer 1, where neuron densities are low. It is possible, therefore, that the failure to reach significance was due to small sample size only. Surprisingly, 6 out of 16 pairs with a result ≥0.75 were also in layer 1.
Overall, the strongest effects were observed in pairs of neurons belonging to the same type (12 out of 16 results with a result ≥0.75) and the mean strength of the effect was more than 2 times higher within the same type than across types  (0.60 vs. 0.26). This suggests that neurons whose axons and dendrites have similar shapes reach overlapping regions of the microcircuit, where they form connections with strongly overlapping sets of partners. Generally, the common neighbor rule was stronger for inhibitory than for excitatory types (0.32 ± 0.27 for I-I pairs, 0.25 ± 0.12 for E-I pairs, 0.23 ± 0.09 for E-E pairs, differences between all 3 groups significant, P < 0.01, two tailed t-test). On this basis, we predict that the excess of common neighbors will be larger in networks involving inhibitory types than in purely excitatory networks.
The neuron types with the strongest tendency to form common neighbors were DBCs in layers 5 and 6, which both displayed results more than 2 times higher than the mean of these layers. Interestingly, this tendency was weaker or absent in DBCs in layers 2/3 and 4. This distinction coincides with significant differences in morphology. The axons of DBCs in layers 2/3 and 4 point predominantly downwards while the dendrites point upwards. In layers 5 and 6, the axons tend to point upwards and the dendrites downwards (see Markram et al., 2015). This is further evidence that the excess of common neighbors is a consequence of neuron morphology.
Another higher-order rule of connectivity found by (Perin et al. 2011) is that the connection probability for pairs of PCs is higher when they have higher numbers of common neighbors. Again, we found the same phenomenon in the NMC model, where the connection probability increased from 0.1, for neurons with no common neighbors, to 0.24, for neurons with 3 common neighbors (Fig. 7A).
To measure the strength of this "common neighbor bias" for pairs of neuron types we used a version of the global "clustering coefficient" (Luce and Perry 1949; Watts and Strogatz 1998).  Figure 6, we evaluated connection probabilities for pairs of PCs in layer 5 with different numbers of common neighbors (gray bars) and compared the results to data from (Perin et al. 2011) (green). In a random network, numbers of common neighbors would have no effect on connection probabilities (black). (B) The global cluster coefficient is computed as the fraction of closed triangle motifs over all triangle motifs. As in Figure 6, we consider triangles where 2 of the corners are formed by neurons of a given pair of types and the third neuron is in the same layer. Triangles are considered closed only if there exists a connection from the presynaptic neuron type (mpre) to the postsynaptic type (mpost ). Connected neuron pairs with many common neighbors will close many triangles simultaneously. Thus, positive correlation between common neighbor numbers and connection probability will increase the clustering coefficient. (C) Mean ratio of the observed clustering coefficient to the clustering coefficient in a randomized network with the same neuron positions and distance-dependent connection probabilities (7 instances of connectivity; 10 randomized per anatomically constrained instance). Nonsignificant differences shown in gray (paired t-test). Briefly, a pair of neurons that share a common neighbor gives rise to a "triangle motif". The clustering coefficient is defined as the fraction of such triangles that are closed, that is, the fraction of triangles where the original pair is connected. The more common neighbors a pair shares, the more triangles are closed when a connection exists. Thus, given an overall connection probability, higher values of the clustering coefficient are equivalent to stronger common neighbor bias. As above, our analysis was connection-type specific, considering only triangles formed by pairs of neuron types and their common neighbors in the same layer (Fig. 7B). Further, we considered connections between the original pair only in the indicated direction, measuring only the increase of directed connection probability "from" a neuron type m pre to a type m post . "Common neighbor bias" was then computed as the ratio between the clustering coefficients in the model and a control network, where the connections between the types under consideration (i.e., → m m pre post ), were shuffled, while maintaining the distance-dependent connection probability profile. Connections from m pre or m post to the rest of the network, and consequently the numbers and identities of common neighbors between m pre /m post pairs, remained untouched.
As before, we found the effect described by (Perin et al., 2011) in all regions of the NMC model (Fig. 7C). In fact, the effect for pairs of PCs in layer 5 (the types in the original study) was weaker than for most other pairs. The strongest effect was found for I → I pairs, albeit with high variability (2.45 ± 2.30). With other connection types, variability was lower (1.83 ± 0.46 for I → E connections, 1.91 ± 0.90 for E → I and 1.49 ± 0.18 for E → E). Between group differences were all statistically significant (P < 0.01) except for I → E against E → I (P = 0.03).
The strength of the common neighbor bias appears to depend more on the presynaptic than on the postsynaptic neuron type (average standard deviation across postsynaptic vs. presynaptic neuron type: 0.90 vs. 0.57; P = 0.024, paired t-test). This is an indication that it depends more on axonal than on dendritic morphology.

The Role of Morphological Variability
The finding that the anatomically constrained connectivity reproduces many nonrandom features of biological connectivity complements previous findings that it reproduces average connection probabilities and a bias for reciprocal connections , observed in many in vitro studies (Song et al. 2005;Larimer and Strowbridge 2008;Perin et al. 2011). Clearly, most of the nonrandom features we observed were strongly dependent on neuron type (Figs 4, 5D, 6, 7), highlighting the crucial role of morphological variability in connectivity. However, it remained unclear how far these features were determined by the morphological characteristics shared by all neurons of a specific type, and how far they depended on individual within-type morphological variability.
To answer this question, we developed an algorithm that connected neurons based on the overlap between the average axonal and dendritic "clouds" of their respective types, that is, the probability distribution of the relative positions of dendrite and axonal segments relative to the soma (see Markram et al. 2015, Methods). In this approach, connectivity was still constrained by morphological variability across neuron types, the positions of neurons within each layer and the maximum number of connections an axon can sustain. However, the approach ignored individual morphological variability within a neuron type as expressed through branching angles, segment length distributions and other measures of local geometry. Thus, types that are very different in terms of their "specific" versus "nonspecific" layouts or the correlations between the positions of individual segments  could still give rise to the same "cloud" representation (see Supplementary Fig. S7).
Briefly, for each pair of neuron types, we computed the convolution of their respective dendritic and axonal clouds yielding a measure of overlap for different relative soma locations (Fig. 8A). Next, for each pair of neurons belonging to a specific pair of types, we looked up the strength of the overlap for their relative positions and placed the value in a Table of overlaps. Finally, we applied a transfer function to the Table (see below) and normalized the results, such that the resulting connection probabilities would conserve the expected total number of connections between the 2 neuron types. We then used the table to generate random instances of "cloud-derived" connectivity for further analysis.
With identity as the transfer function, the distance dependency of connection probabilities were much flatter than in the NMC model, that is, there were too few connections at low distances and too many at high distance (Fig. 8B, green lines vs. dashed, black lines). In fact, the profiles were comparable to those obtained by randomly pruning appositions, without applying a multi-synapse rule, that is, by ignoring anatomical constraint 1 (Fig. 8B, dotted lines).
A multi-synapse rule will make anatomically strong connections more likely and weak connections less likely. To emulate this effect, we recomputed the connectome using alternative transfer functions y = O 2 and y = O 3 (Fig. 8B, blue, red lines). With y = O 2 , we found a good match between the distance dependency profile for the cloud-derived connectome and the profile for the actual microcircuit model (Fig. 8C), the only exceptions being a few connection types involving MCs, where y = O 3 provided a better fit. More generally, we found optimal fits for exponents around 2 ± 0.2 (Fig. 8D), the only exceptions being some of the connection types formed by MCs onto PCs. These results suggest that while the bias toward structurally stronger connections may be type specific, the realistic distance-dependent connection probability profiles in the NMC model depend on the large-scale morphological properties of the neuron types and not on the shapes of individual neurons within a type.
As a further test, we compared the cloud-derived connectome (transfer function y = O 2 ) to the full-detail NMC model, in terms of bias for reciprocal connectivity, that is, 8E). Once more we found an exceptional match, except for pathways involving MCs (Fig. 8E, red and maroon), where higher exponents produced a better fit (Fig. 8F). Thus, bias for reciprocal connectivity is also independent of the morphologies of individual neurons. However, this is not the case for common neighbor bias. In fact, there is connectivity, that is, practically no overlap between the value of the normalized cluster coefficient for the cloud-based connectivity, calculated as in Figure 7, and the value for the full-detail microcircuit model (Fig. 8G). Only the very largest coefficient in the cloudderived connectivity (1.39, L4_PC to L4_MC) exceeds the minimal value for the full-detail model (1.3, L5_STPC to L5_TTPC1). Even using transfer functions with higher exponents, the common neighbor bias in the cloud-based connectivity remained lower (Fig. 8H, 4 exemplary connection types shown; results for others were similar). This is direct evidence that the common neighbor bias depends on individual differences between the morphologies of neurons belonging to the same type.

Discussion
This study explored the extent to which morphological diversity across and within neuronal types constrains connectivity. We used known biological rules to reconstruct connectivity between known neuron types in different statistical instances of a biologically driven in silico model of neocortical microcircuitry. We found on the one hand, that the average morphology of neuronal types imposes significant constraints on the first order structure of the network. These are reflected in the strengths of connections between different neuron types, interlaminar connectivity, distance-dependent connectivity (which varied between neuron types) and reciprocal connectivity bias. On the other hand, higher-order structure, such as the experimentally derived common neighbor bias of connectivity, depends on morphological diversity within individual neuronal types. The exact nature of the morphological features leading to this effect-possibly distributions of branch angles/lengths or correlations thereinremains an open question to be addressed in future work. Our approach to the analysis of local microcircuits allowed us to generate numerous predictions about the structure of their connectivity. For example, the analysis predicts that different connection types have different distance-dependent connectivity profiles (Fig. 4), that synaptic innervation follows statistical rules (Fig. 5, see Supplementary Fig. S6) and that almost all neuron types display clustered connectivity (Figs 6, 7). These predictions demonstrate the power of statistical, in silico connectome derivations to complement in vivo, in vitro, and EM measurements.
The accuracy of the predictions depends on the underlying modeling method, and the biological data used to parameterize it. Although the integration process exploited an unprecedented volume of biological data, these data were still not complete, making it necessary to base some aspects of the model on extrapolation.
In particular, the model captured morphological variation within neuron types by creating multiple new samples of manually reconstructed neuron morphologies, adding small statistical variations to the branch lengths and angles of the original morphology. It is possible that this process underestimated the morphological diversity within cell types actually present in cortical tissue. Based on the finding that this intrinsic morphological diversity shapes higher order connectivity, it is likely that the higher order connectivity is even more complex than predicted.
Since different neuron types play an interdependent role in determining connectivity, it is possible that the addition of new neuron types to the model would affect the connectivity of other types. However, our main conclusions-that first order structure depends on morphological diversity between cell types, that higher order connectivity depends on diversity within types, that almost all types of neurons will show strong common neighbor bias-are more likely to be strengthened than weakened by the inclusion of additional morphological detail.
The fact that anatomical constraints alone are enough to account for a wide range of known features of neuronal connectivity (except for specifically crafted patterns that are cell-type specific, such as the innervation of the axon initial segment by Chandelier neurons), suggests that it may be possible to characterize biologically available configurations of connectivity, that is, the configurations reachable via structural plasticity, with only few-if any-additional constraints. If this is so, the upper bound placed by anatomical constraints on the storage capacity of neural microcircuitry provides a useful estimate of its limits.
The same observations suggest that the activity-dependent long-term microcircuit plasticity or rewiring that has been demonstrated experimentally (Le Be and Markram 2006) is not the main cause of the clustered connectivity we observe in the brain, but rather a mechanism to shift from one specific clustering configuration to another. Despite anatomical constraints, the space of possible configurations is still enormous, and it is unlikely that plasticity can explore it completely. We suggest, therefore, that for learning to be feasible, plasticity mechanisms must necessarily place significant global constraints on how connections can change. One way of achieving this would be through homeostatic plasticity, for example, regulation of neuron firing rates (Renart et al. 2003;Turrigiano and Nelson 2004;Delattre et al. 2015). Mechanisms that require continuous shifting between a vast set of possible configurations pose significant challenges for current theories of information processing, storage and retrieval. It is likely, therefore, that future studies will have to consider the stability of individual configurations in addition to their availability.

Supplementary Material
Supplementary data are available at Cerebral Cortex online. The in silico connectome derived in this study can be freely obtained from the Neocortical Microcircuit Collaboration Portal -https:// bbp.epfl.ch/nmc-portal/downloads .