Increasing evidence shows that phenotypic variance is genetically determined, but the underlying mechanisms of genetic control over the variance remain obscure. Here, we conducted variance-association mapping analyses to identify expression variability QTLs (evQTLs), i.e. genomic loci associated with gene expression variance, in humans. We discovered that common genetic variants may contribute to increasing gene expression variance via two distinct modes of action—epistasis and destabilization. Specifically, epistasis explains a quarter of the identified evQTLs, of which the formation is attributed to the presence of ‘third-party’ eQTLs that influence the gene expression mean in a fraction, rather than the entire set, of sampled individuals. On the other hand, the destabilization model explains the other three-quarters of evQTLs, caused by mutations that disrupt the stability of the transcription process of genes. To show the destabilizing effect, we measured discordant gene expression between monozygotic twins, and estimated the stability of gene expression in single samples using repetitive qRT-PCR assays. The mutations that cause destabilizing evQTLs were found to be associated with more pronounced expression discordance between twin pairs and less stable gene expression in single samples. Together, our results suggest that common genetic variants work either interactively or independently to shape the variability of gene expression in humans. Our findings contribute to the understanding of the mechanisms of genetic control over phenotypic variance and may have implications for the development of variance-centred analytic methods for quantitative trait mapping.