We study the modular symmetry in four-dimensional low-energy effective field theory, which is derived from type IIB magnetized D-brane models and type IIA intersecting D-brane models. We analyze modular symmetric behaviors of perturbative terms and non-perturbative terms induced by D-brane instanton effects. Anomalies are also investigated and such an analysis on anomalies suggests corrections in effective field theory.

## 1. Introduction

T-duality in string theory relates a theory with the compact space size $R$ to another theory with the size $1/R$. Thus, T-duality is a quite non-trivial symmetry in string theory. Indeed, one type of superstring theory is related to a different type of superstring theory by T-duality. (For a review, see Ref. [1].)

T-duality also has a remnant in four-dimensional (4D) low-energy effective field theory derived from superstring theory. In particular, 4D low-energy effective field theory of heterotic string theory with a certain compactification is invariant under the modular transformation of the moduli $\tau $,

The modular symmetry plays an important role in studies on 4D low-energy effective field theory of heterotic string theory. For example, moduli stabilization and supersymmetry breaking were studied with the assumption that non-perturbative effects are also modular invariant [2,3]. Moreover, anomalies of this symmetry were analyzed [4,5]. The anomaly structure in heterotic string theory has a definite structure.^{1} Their phenomenological applications were also studied (see, e.g., Refs. [7,8]). In addition, the modular invariant potential of the modulus was studied for cosmic inflation [9]. Thus, the modular symmetry in 4D low-energy effective field theory is important from several viewpoints: theoretical, particle physics, and cosmology.

In this paper, we study the modular symmetry in 4D low-energy effective field theory derived from type II superstring theory. In particular, we consider the 4D low-energy effective field theory derived from type IIB magnetized D-brane models and type IIA intersecting D-brane models. Their 4D low-energy effective field theories have been studied before (for a review, see Refs. [10,11]). We study the modular symmetry at perturbative level in their low-energy effective field theories. The T-duality of Yukawa couplings between magnetized D-brane models and intersecting D-brane models was studied in Ref. [12]. This is very useful for our purpose. We extend such analysis to show modular transformation of 4D low-energy effective field theory including three-point and higher-order couplings. Also, their anomalies are examined and the anomaly structure could provide non-trivial information like those in heterotic string theory. Furthermore, we discuss non-perturbative effects.

The paper is organized as follows. In Sect. 2, we study the modular symmetry of Yukawa couplings and higher-order couplings at the perturbative level in the 4D low-energy effective field theory derived from type IIB magnetized D-brane models. In Sect. 3, we study supergravity theory derived from type IIA intersecting D-brane models. In particular, we investigate the anomaly structure of the modular symmetry. In Sect. 4, we study the modular symmetry of non-perturbative terms induced by D-brane instanton effects. Section 5 provides the conclusions.

## 2. Modular symmetry

Here, we study the modular symmetry in the 4D low-energy effective field theory derived from type IIB magnetized D-brane models.

### 2.1. Magnetized D-brane models

We start with magnetized D9-brane models in type IIB theory. We compactify six-dimensional (6D) space to the 6D torus, e.g. three 2-tori. The metric of the $r$th 2-torus for $r=1,2,3$ is written by

#### 2.1.1. Yukawa couplings

Here, we review the analysis of Yukawa couplings in Ref. [12]. Our setup includes several stacks of D9-branes with magnetic fluxes. We assume that our setup preserves 4D $N=1$ supersymmetry. Among several D-branes, we consider two stacks of $Na$ and $Nb$ D9-branes, which correspond to the $U(Na)\xd7U(Nb)$ gauge symmetry. We put magnetic fluxes, $Fra(=Fzrz\xafra)$ and $Frb(=Fzrz\xafrb)$ on these D-branes along $U(1)a$ and $U(1)b$ directions of $U(Na)=U(1)a\xd7SU(Na)$ and $U(Nb)=U(1)b\xd7SU(Nb)$. The magnetic fluxes must be quantized as $Fra=\pi iIm\u2009\tau rmar$ in the complex basis. For simplicity, we do not include Wilson lines here [12].

The open strings between these magnetized branes have massless modes. There appear $Iabr$ zero-modes on the $r$th 2-torus, where $Iabr=mar\u2212mbr$, and the total number of massless modes is given by their product, $Iab=\u220fr=13Iabr$. Their zero-mode profiles on the $r$th 2-torus are written by [12]

These zero-modes are also written by another basis,

Note that the zero-mode profiles of bosonic and fermionic modes are the same in supersymmetric models. For $N=Iabr<0$, the zero-mode profiles are obtained by $\psi j,N(\tau r,zr)*$.

In addition to the above two stacks of D-branes, we consider another stack of $Nc$ D9-branes. Then, there appear three types of massless modes, $a\u2212b$, $b\u2212c$, and $c\u2212a$ modes. Their Yukawa couplings among canonically normalized fields can be obtained by overlap integral of wavefunctions,

Similarly, the Yukawa couplings can be written in the basis $\chi $,

It would be convenient to use the 4D dilaton,

#### 2.1.2. Modular symmetry

Now, let us study the modular transformation of the complex structure moduli $\tau r$. Recall that we use the basis, so that the fields are normalized canonically. Thus, we just investigate the modular transformation of the Yukawa couplings. The modular transformation (1) is generated by the two generators, $s$ and $t$,

The $\u03d1$-function part in the Yukawa coupling is transformed under $s:\tau \u2192\u22121/\tau $,

The above results can be extended to the magnetic flux,

#### 2.1.3. Higher-order couplings

We can study higher-order couplings in a similar way [13]. For example, the four-point coupling can be obtained by computing the integral of zero-mode profiles,

Similarly, we can compute generic $n$-point couplings [13], whose $\tau $ dependence as well as $\varphi 4$ dependence appears in the form

Similarly, we can study the orientifold and orbifold compactifications. For example, the zero-mode profiles on the $Z2$ orbifold can be written by linear combinations of zero-mode profiles on the torus [14],

## 3. Supergravity and anomaly

In this section, we study modular symmetry within the framework of string-derived supergravity and investigate its anomaly.

### 3.1. Intersecting D-brane models

In the previous section, we studied modular symmetry in 4D low-energy effective field theory of magnetized D-brane models for canonically normalized fields. Here, we study type IIA intersecting D-brane models, which are T-dual to magnetized D-brane models. In intersecting D-brane models, the Kähler metric of matter fields was computed [10,15–18]. In this section, we study the modular symmetry from the viewpoint of supergravity derived from intersecting D-brane models. In particular, we study intersecting D6-brane models, where two sets of D6-branes, e.g. $D6a$ and $D6b$, intersect each other at an angle $\pi \theta abr$ on the $r$th 2-torus.

First, we write the supergravity fields in type IIB theory as

We take the T-dual of the Yukawa coupling (13) of the magnetized D9-brane models, and then we can write the Yukawa coupling of intersecting D-brane models:

Within the framework of supergravity, physical Yukawa couplings are written by

The Kähler metric of matter fields has been computed [10,15–18]. The Kähler metric of the $a\u2212b$ sector would be written as

### 3.2. Anomaly

In the previous section, the modular symmetry in the supergravity basis was studied. The chiral multiplet $\Phi ab$ in the $a\u2212b$ sector has the Kähler metric (34). Thus, the chiral multiplet, $\Phi ab$, transforms as

Such a modular transformation may be anomalous. The supergravity Lagrangian includes the following couplings:

These couplings induce the anomaly of modular symmetry. Its anomaly coefficient of mixed anomaly with the $SU(Na)$ gauge group is written by [4]

This anomaly can be canceled in two ways [4,5]. One is moduli-dependent threshold corrections and the other is the generalized Green–Schwarz mechanism. The latter would lead to mixing of moduli, e.g. in the Kähler potential. In order to see this, we first review briefly anomalous $U(1)$ and the Green–Schwarz mechanism in the next subsection [10,11,19].

#### 3.2.1. Anomalous $U(1)$

First, let us consider the $D6b$-branes wrapping the 3-cycle $[\Pi b]$, whose wrapping numbers are $(nbr,mbr)$ along $(xr,yr)$. We introduce the basis of 3-cycles, $[\alpha 0]$ and $[\alpha k]$ with $k=1,2,3$, such that $[\alpha 0]$ is along $(1,0)$ for all of $(xr,yr)$, while $[\alpha k]$ is along $(1,0)$ only for $r=k$ and $(0,1)$ for the others. We also introduce their duals $[\beta k]$ such that $[\alpha i]\u22c5[\beta k]=\delta ik$. These D6-branes correspond to the $U(Nb)$ gauge group, and its gauge kinetic function $fb$ is written by

Now, we study the $U(1)a\u2212SU(Nb)2$ mixed anomaly. Its anomaly coefficient can be written by

The Green–Schwarz mechanism is the same in the toroidal, orientifold, and orbifold compactifications.

#### 3.2.2. Anomaly cancelation of modular symmetry

As mentioned above, the modular anomaly can be canceled in two ways [4,5]. One is moduli-dependent threshold corrections and the other is the generalized Green–Schwarz mechanism. In general, the gauge kinetic function has one-loop threshold corrections due to massive modes as

The transformation (54) implies that the Kähler potential is not invariant under the modular transformation. The Kähler potential must be modified as

^{2}

Here, we return to the type IIB model studied in Sect. 2. Similar to the above, we may need to replace

## 4. D-brane instanton effects

In Sect. 2, we studied the modular symmetry of perturbative terms in the Lagrangian. In this section, we study terms due to non-perturbative effects, in particular terms induced by D-brane instanton effects. First, we study an illustrative example, and then we will discuss generic aspects.

### 4.1. Example

In this subsection, we study a Majorana mass term induced by an E5-brane in type IIB magnetized orientifold models with O9-planes compactified on a $Z2\xd7Z2\u2032$ torus. In these models, the non-perturbative corrections to the superpotential are written as [22,23]^{3}

### 4.2. Generic discussion

The example in the previous subsection shows the modular symmetry of non-perturbative terms induced by D-brane instanton effects for the complex structure moduli in type IIB magnetized\break D-brane models. Moreover, this example suggests a generic aspect. The D-brane instantons induce the non-perturbative terms such as

^{4}Here, $Vol(E5)$ denotes the volume of the D-brane instanton in the compact space, and it depends only on $Ar$, but not $\tau $. Furthermore, $y(n)$ denotes the couplings among zero-modes and 4D fields $\Phi i$, and these are computed in the same way as the perturbative couplings shown in Sect. 2. The $\tau $ dependence appears only through these couplings $y(n)$. Therefore, terms induced by D-brane instanton effects are also modular symmetric.

In this section, we have not taken into account the moduli mixing so far. However, the discussion in Sect. 3 would lead to modification such as Eq. (61).

## 5. Conclusion

We have studied the 4D low-energy effective field theory, which is derived from type IIB magnetized D-brane models and type IIA intersecting D-brane models. We have studied the modular symmetric behavior of perturbative terms. Also, such analysis has been extended to non-perturbative terms induced by D-brane instanton effects. We have also investigated the anomaly of the modular symmetry. Its cancelation would require moduli mixing correction terms in low-energy effective field theory. Thus, the modular symmetry is important in understanding the 4D low-energy effective field theory of superstring theory.

## Acknowledgements

T. K. and S. U. are supported in part by Grants-in-Aid for Scientific Research No. 26247042 and No. 15J02107 from the Ministry of Education, Culture, Sports, Science, and Technology in Japan.

## Funding

Open Access funding: SCOAP^{3}.

## References

^{1}See also Ref. [6].

^{3}For explicit computations on intersecting D-brane orbifold models, see e.g. Ref. [24].

^{4}More precisely, the coefficient $C$ may include a functional determinant of the Dirac operator as well as a bosonic Laplacian operator produced by the integration of massive modes [22,25]. However, these coefficients are canceled if the SUSY is not broken. Even if the SUSY is broken, eigenvalues of the Dirac operator and Laplacian operator depend only on $Ar$, but they are independent of the complex structures [12]. Thus our conclusion would not be affected by this coefficient.

^{3}