The effect of the warping term on the fractional quantum Hall states in topological insulators

The warping effect on the fractional quantum Hall (FQH) states in topological insulators is studied theoretically. Based on the perturbed wavefunctions, which include contributions from the warping term, analytical expressions for Haldane's pseudopotentials are obtained. We show that the warping term does not break the symmetry of the pseudopotentials for $n$=$\pm1$ Landau levels (LLs). With increasing the warping strength of the Fermi surface, our results indicate that the stability of the FQH states for LL $n=0$ (LLs $n$=$\pm1$) becomes stronger (weaker), and the excitation gap at $\nu=1/3$ FQH state for LL $n$=0 also increases while the gaps for LLs $n$=$\pm1$ are unchanged.

Topological insulators (TIs) as a new phase of quantum matter, which can not be adiabatically connected to conventional insulators and semiconductors, have been intensively studied in recent years [1][2][3][4][5][6][7][8][9][10]. TIs are characterized by a full insulating gap in the bulk and protected gapless edge or surface states. Near the Fermi level the low-energy dispersion of the TI surface states shows a Dirac linear behavior. However, the recent angle-resolved photoemission spectroscopy experiments show that the Fermi surface in Bi 2 Te 3 [9,10], a typical TI, is a snowflake shape rather than a circle one. The origin of this snowflake-like Fermi surface has been confirmed to arise from an unconventional hexagonal warping term [11]. It is this warping term that brings about many unique physical phenomena [12][13][14], which can not be observed in other systems, including the extensively studied graphene and the conventional two-dimensional electron gas.
More recently, there has been emerging attention to the interactions of the Dirac-type quasiparticles and their strong correlation effects in TI, especially the TI fractional quantum Hall (TIFQH) states. Despites no undeniable experimental observation of the TIFQH states heretofore, some theoretical studies have been undertaken. For example, DaSilva [15] predicted the stability of the TIFQH states for Landau levels (LLs) with index n=0 and ±1 in TIs. Apalkov and Chakraborty studied the finite thickness effect on the TIFQH states [16]. Also, the present authors investigated the influences of the Zeeman splitting and the tilted strong magnetic field on the stability of the TIFQH states [17,18] with large g factor. However, many important and interesting open questions, such as the warping effect, the spin excitations, and the subband-LL coupling, have not been discussed in the TIFQH regime.
In this paper we theoretically study the warping effect on the TIFQH states. Here the warping term is perturbativly treated. With the help of the numerical calculations, we show that the warping term can not break the symmetry of the Haldane's pseudopotentials for n=±1 LLs, which differs from the role of the spin splitting [17]. Moreover, our results indicate that with the increase of the warping strength of the Fermi surface, the stability of the TIFQH states for LL n = 0 (LLs n=±1) become stronger (weaker), and the excitation gap at ν = 1/3 filling for LL n=0 (gaps for LLs n=±1) also increases (keep unchanged).
In the presence of a perpendicular magnetic field B = Bẑ, the effective Dirac Hamiltonian for Bi 2 Te 3 (111) surface is written as where Π = k+eA/c with the wave vector k = (k x , k y , 0) and the gauge A = B(−y/2, x/2, 0).
Here, Π ± = Π x ±Π y , σ x,y,z are Pauli matrices, v f denotes the Fermi velocity, and λ describes the hexagonal warping strength of the Fermi surface [11]. Here we have assumed that the Zeeman splitting is much weaker than the warping term and therefere can be neglected for the first step in order to solely illustrate the role played by the warping term. This is the case for Bi 2 Te 3 (111) system. By introducing the ladder operators a † = 1 When λ=0 the Hamiltonian (2) can be exactly solved, and the eigenstates are given by where the symbol |n, m is the non-relativistic two-dimensional electron gas Landau eigenstates with non-relativistic quadratic dispersion relation in nth LL with angular momentum m. The corresponding LLs are expressed as ε n = sgn (n) v f 2 |n|/l B . When the warping term is taken into account (λ = 0), however, the single-particle eigenstate can not be obtained directly. Fortunately, one can take the perturbation method to get eigenstates of Hamiltonian (2) since the warping term, √ 2λl −3 B , is much smaller than the typical energy space between the nearest-neighboring LLs, After some long but straightforward algebraic operations, and only keeping the first-order terms, we where the coefficients are In the following discussion we will focus our attention to the TIFQH states of |n| 1, because the stable TIFQH states can only be observed for LLs |n| 1 [15]. The Haldane's pseudopotential [19] for Coulomb interaction V (r) = e 2 ǫr between electrons in the nth LL with relative angular momentum m is given by in terms of Laguerre polynomials L m (x) and the form factor F (q) = Ψ n |e −iq·η |Ψ n with the cyclotron variable η = r − R. Here, R is the guiding-center position. Explicitly, for LLs |n| 1 we have   Fig. 1), one can clearly find that in the presence of the warping term, the magnitude of the pseudopotentials increases (red stars in Fig. 1).
Subsequently, the stability of the TIFQH states should also be modified by the warping term. The typical results of V (n,1) /V (n,3) (n=0 and n=±1) are shown in Fig. 2  In what follows, by using the exact diagonalization method in the spherical geometry, we investigate the system with the fractional filling factor ν=1/(2p + 1), where p is an integer.
For briefness we only illustrate the ν=1/3 TIFQH state, which is realized at S = 3 2 (N − 1) in the spherical geometry with N being the electron number. Under this configuration, the perpendicular magnetic field is equivalent to a fictitious radial magnetic field produced by a magnetic monopole at the center of a sphere of radius R = √ Sl B (in unit of flux quanta), and the many-body states could be described by the total angular momentum L and its z component L z .
We show in Fig. 3 the energy spectra of the many-body states at ν=1/3 filling for N=7 electrons. Comparing the two cases with (ζ=0.2) and without (ζ=0) warping term, one can see from Fig. 3 that the gap width at n=0 LL between the ground state and the first excited state has a visible increment while those at n=±1 LL keep unchanged. Furthermore, we calculate the corresponding excitation gap width E n g (ζ) by increasing the warping term from ζ=0 to ζ=0.3. The variation ∆E g = E n g (ζ) − E n g (ζ = 0) as a function of ζ are plotted in Fig. 4, which shows that the TIFQH gap between the ground state and the lowest excited state at n=0 LL (solid line) are sensitively dependent on the warping term while ∆E g at n=±1 LLs (dashed line) keeps a constant no matter the warping term is included or not.
This also implies that the warping term is different from the Zeeman splitting and the tilted magnetic field, which will induce a change in the gap of the TIFQH states at LLs n=±1 [17,18].
In summary, we perturbatively studied the effect of the warping term on the TIFQH states. It was found that differing from the role of the Zeeman splitting and the tilted magnetic field, the warping term does not break symmetry of the Haldane's pseudopotentials for n=±1 LLs. Our results showed on one hand that, the stability of the ν = 1/3 TIFQH states for LL n = 0 (LLs n=±1) become stronger (weaker) by increasing the warping strength of the Fermi surface. On the other hand, the excitation gap for LL n=0 increases with increasing the strength of the warping term, while the gaps for LLs n=±1 are yet insensitive to this term.