Meta-analysis of genome-wide association studies has resulted in the identification of hundreds of genetic variants associated with growth and stature. Determining how these genetic variants influence growth is important, but most are non-coding, and there is little understanding of how these variants contribute to adult height. To determine the mechanisms by which human variation contributes to growth, we combined spatial genomic connectivity (high-throughput conformation capture) with functional (gene expression, expression Quantitative Trait Loci) data to determine how non-genic loci associated with infant length, pubertal and adult height and contribute to gene regulatory networks. This approach identified intergenic single-nucleotide polymorphisms (SNPs) ∼85 kb upstream of FBXW11 that spatially connect with distant loci. These regulatory connections are reinforced by evidence of SNP-enhancer effects and altered expression in genes influencing the action of human growth hormone. Functional assays provided evidence for enhancer activity of the intergenic region near FBXW11 that harbors SNP rs12153391, which is associated with an expression Quantitative Trait Loci. Our results suggest that variants in this locus have genome-wide effects as key modifiers of growth (both overgrowth and short stature) acting through a regulatory network. We believe that the genes and pathways connected with this regulatory network are potential targets that could be investigated for diagnostic, prenatal and carrier testing for growth disorders. Finally, the regulatory networks we generated illustrate the power of using existing datasets to interrogate the contribution of intergenic SNPs to common syndromes/diseases.