We investigate the dispersion in the internal colours of faint spheroidals in the Hubble Deep Fields North and South. In high-redshift rapid-collapse scenarios, the dispersion in internal colours should be small at moderate redshift apart from a small metallicity induced reddening in the enriched cores. However, recently assembled spheroidals are likely to show non-homologous internal colours, at least until younger stellar populations become fully mixed. Here we find that a remarkably large fraction (≳30 per cent) of the morphologically classified spheroidals with I814W<24 mag show strong variations in internal colour, which we take as evidence for recent episodes of star formation. In most cases these colour variations manifest themselves via the presence of blue cores, an effect of opposite sign to that expected from metallicity gradients. Examining similarly selected ellipticals in five rich clusters with 0.37<z<0.83 we find a significant lower dispersion in their internal colours. This suggests that the colour inhomogeneities have a strong environmental dependence being weakest in dense environments where spheroidal formation was presumably accelerated at early times. We use the trends defined by the cluster sample to define an empirical model based on a high redshift of formation and estimate that at z∼1 about half the field spheroidals must be undergoing recent episodes of star formation. Using spectral synthesis models, we construct the time dependence of the density of star formation (ρSFR). Although the samples are currently small, we find evidence for an increase in ρSFR between z=0 and 1. We discuss the implications of this rise in the context of that observed in the similar rise in the abundance of galaxies with irregular morphology. Regardless of whether there is a connection our results provide strong evidence for the continued formation of field spheroidals over 0<z<1.