Irregular parameter dependence of numerical results in tensor renormalization group analysis

Abstract

We study the parameter dependence of numerical results obtained by the tensor renormalization group. We often observe irregular behavior as the parameters are varied with the method. Using the two-dimensional Ising model we explicitly show that the sharp cutoff used in the truncated singular value decomposition causes this unwanted behavior when the level crossing happens between singular values below and above the truncation order as the parameters are varied. We also test a smooth cutoff, instead of the sharp one, as a truncation scheme and discuss its effects.

1. Introduction

The tensor renormalization group (TRG) is a promising approach that can solve the sign problem inherent in Monte Carlo simulations. Since it was proposed in the two-dimensional Ising model [1], many studies have been carried out for various models in lattice field theory [2–14]. In TRG, the truncated singular value decomposition (SVD) is used to define a coarse-grained tensor, which is given in the manner of a sharp cutoff such that the |$D_{\mathrm{cut}}$| largest singular values and corresponding singular vectors are kept and the others are discarded. Although the cutoff yields possible systematic errors, it is expected that the result should converge to the correct value as |$D_{\mathrm{cut}}$| increases.

The results using the TRG, however, do not smoothly depend on the parameters in the theory. They often show irregular behavior at some parameters off the critical point, which can be misidentified as a physical phenomenon such as an unknown phase transition. It also becomes an issue in evaluating the numerical derivative with respect to the parameter at the irregular point, for instance in evaluating the internal energy by the numerical derivative of the free energy. We can of course obtain a satisfactory result for a simple model such as the two-dimensional Ising model taking a sufficiently large |$D_{\mathrm{cut}}$| to avoid such misbehavior. However, it is difficult to increase |$D_{\mathrm{cut}}$| for general lattice theories with multi-dimensional fields, so it is important to understand and avoid the irregular behavior of the results.

In this letter we investigate the origin of the irregular parameter dependence shown in the TRG results. We present some numerical evidence that it is caused by the level crossing between singular values within and beyond the sharp truncation as the parameters are varied. In this sense the irregular behavior is inevitable for the TRG method with the sharp cutoff. In order to show that the irregular behavior is not a physical phenomenon and to obtain a clue to improving the behavior, we also test other cutoff schemes such as a smooth cutoff.

The rest of the letter is organized as follows. In Sect. 2 we review the standard TRG method with the sharp cutoff in the two-dimensional Ising model with sample numerical results. The mechanism of the irregular behavior is explained in detail with some numerical evidence in Sect. 3. We also test other cutoff schemes. Our conclusions are summarized in Sect. 4.

2. TRG in the two-dimensional Ising model

We briefly review the TRG method in the two-dimensional Ising model. We consider a two-dimensional square lattice whose sites are labeled by |$n=(n_1,n_2)$| for |$n_1,n_2 \in \mathbb{Z}$|⁠. The spin variable |$\sigma_n$| assigned to site |$n$| takes the discrete values |$\sigma_n \in \{1,-1\}$|⁠. The two-dimensional Ising model is then defined by the Hamiltonian

\begin{align} {\cal H}=-J \sum_{\langle i,j \rangle}\sigma_{i}\sigma_{j}, \end{align}

(1)

where |$\langle i,j \rangle$| denotes possible pairs of nearest-neighbor sites and |$J$| is the coupling constant.

The partition function |$Z={\rm Tr}\, {e}^{-\beta {\cal H}}$| with the inverse temperature |$\beta=1/T$| can be expressed as a tensor network form:

\begin{eqnarray} Z=\sum_{i,j,k,l,\dots} T_{ijkl} \ldots , \label{tensor_representation} \end{eqnarray}

(2)

where

\begin{eqnarray} \label{initial_tensor} T_{ijkl} = e^{\beta J(ij + j k + kl + li)} \end{eqnarray}

(3)

for |$i,j,k,l=-1,1$|⁠.

Let us denote the bond dimension of |$T_{ijkl}$| as |$N$| for the sake of argument. Note that the initial tensor of Eq. (3) is defined with |$N=2$|⁠. We apply the truncated SVD to |$T_{ijkl}$|⁠:

\begin{align} T_{ijkl} & \approx \sum_{m=1}^{D_{\mathrm{cut}} }U_{(ij)m}\lambda_{m}V^{\dagger}_{m(kl)} , \label{SVD1} \end{align}

(4)

\begin{align} T_{ijkl} & \approx \sum_{m=1}^{D_{\mathrm{cut}} }U^{\prime}_{(li)m}\lambda^{\prime}_{m}{V^{\prime}}^{\dagger}_{m(jk)}, \label{SVD2} \end{align}

(5)

where |$T_{ijkl}$| is treated as a matrix with the column |$(ij)$| and row |$(kl)$| in Eq. (4) and a matrix with the column |$(li)$| and row |$(jk)$| in Eq. (5). The above expressions assume the case of |$N^2 > D_{\mathrm{cut}}$|⁠, while |$D_{\mathrm{cut}}$| in Eqs. (4) and (5) is replaced by |$N^2$| for |$N^2 \le D_{\mathrm{cut}}$| without any truncation. We apply the decomposition of Eq. (4) to the tensors at even sites defined by mod|$(n_1+n_2 , 2) = 0$| and that of Eq. (5) to those at odd sites with mod|$(n_1+n_2, 2) = 1$|⁠. Here, |$U,V,U^{\prime},V^{\prime}$| are unitary matrices and |$\lambda_m$| and |$\lambda'_m$| are singular values that are sorted in descending order.

We immediately find that the expression of Eq. (2) can be approximated as

\begin{eqnarray} Z \approx \sum_{i,j,k,l,\dots} T^{\rm new}_{ijkl}\ldots, \label{new_tensor_representation} \end{eqnarray}

(6)

where

\begin{eqnarray} \label{new_tensor} T^{\rm new}_{ijkl}= \sqrt{\lambda_i \lambda^\prime_j \lambda_k \lambda^\prime_l} \sum_{a,b,c,d=1}^{D_{\mathrm{cut}}} U_{(ab)i} U^{\prime}_{(bc)j} V^{\dagger}_{(cd)k} V^\dagger_{(da)l}. \end{eqnarray}

(7)

Note that the number of tensors decreases because an old tensor is decomposed into two unitary matrices |$U$| and |$V$| (or |$U'$| and |$V'$|⁠) and then four unitary matrices are assembled into a new tensor.

After repeating the above procedures, the tensor network is finally reduced to a single tensor. Taking the appropriate trace for its indices we obtain the approximate value of |$Z$| with |$D_{\mathrm{cut}}$|⁠. The numerical cost of this algorithm is |$O(D_{\mathrm{cut}}^{6})$|⁠, which comes from the computations of Eqs. (4) and (5).

The TRG as described above is a powerful tool for studying two-dimensional lattice models. Although the exact value is obtained in the |$D_{\mathrm{cut}} \rightarrow \infty$| limit, we can reach a sufficient level of accuracy with a moderate value for |$D_{\mathrm{cut}}$| in practical computations. We present a couple of representative results in the TRG analysis for the two-dimensional Ising model on a |$V=2^{16}\times 2^{16}$| lattice as preparation for the study explained in the following section.

The numerical value of the partition function |$Z$| is obtained by repeating the renormalization step of the TRG with a given value of |$D_{\mathrm{cut}}$|⁠. Then the Helmholtz free energy |$F$| is also obtained using |$F=-T\log(Z)$|⁠. The critical temperature |$T_{\rm c}$| is determined from the peak position of the specific heat |$C_V$| obtained by the numerical derivative of |$Z$| with respect to |$\beta$| as |$C_V= -\beta^2\frac{\partial^2}{\partial \beta^2} \log Z$|⁠.

Figure 1 shows the temperature dependence of the free energy density. The black curve denotes the exact solution given in Refs. [15,16], and the black dotted line denotes the critical temperature. As is clearly seen in the figure, the TRG results approach the exact solution as the value of |$D_{\mathrm{cut}}$| increases. The results with |$D_{\mathrm{cut}} \geq 4$| reproduce the exact solution within an error of the order of |$10^{-5}$|⁠.

Fig. 1.

|$T$| dependence of the free energy density evaluated on a |$V=2^{16}\times 2^{16}$| lattice.

Figure 2 shows the |$D_{\mathrm{cut}}$| dependence of the critical temperature. The numerical results fluctuate around the exact solution |$T_{\rm c}^{\rm exact}=2/[\log(1+\sqrt{2})]=2.2691853\ldots$| It is clear that taking a larger value of |$D_{\mathrm{cut}}$| makes the results approach the exact one.

Fig. 2.

Critical temperature evaluated on a |$V=2^{16} \times 2^{16}$| lattice. The computational results are presented as black points, and a dotted line denotes the exact value of the critical temperature |$T_{\mathrm{c}}^{exact} = 2/[\log(1+\sqrt{2})]=2.2691853\ldots$|⁠.

3. The irregular parameter dependence of TRG and a new scheme with a smooth cut

The numerical results of the TRG often show irregular behavior as the parameters are varied. Here we consider the reason why the numerical results do not smoothly depend on the parameters. For simplicity, the numerical computations are performed on |$V=(16)^2$| in this section.

Figure 3 shows the relative residue of the free energy, which is given by the relative difference between the results of the TRG and the exact solution. The irregular behavior is observed as the abrupt jump of the results at several temperatures off the critical point denoted by the black dotted line. For instance, as magnified in the small figure, the result with |$D_{\mathrm{cut}}=12$| shows an irregular jump at |$T_{\rm ref} \approx 2.6075$|⁠. Similar behaviors are observed for other models and other RG methods [6,9,13,17].

Fig. 3.

|$T$| dependence of the relative residual of the Helmholtz free energy, which is evaluated for |$D_{\mathrm{cut}}=8$|⁠, 12, 16, and 20 on a |$V=(16)^2$| lattice. The gray band in the enlarged figure at the bottom right denotes the region where the irregular behavior is caused.

The irregular behavior causes a difficulty in evaluating observables such as the internal energy. Figure 4 shows the relative residue of the internal energy. As shown in the figure, the internal energy does not behave as a smooth function of |$T$| since it is computed by the numerical derivative of |$F$|⁠. Roughly speaking, the step-function-like behavior in the free energy yields delta-function-like behavior in the internal energy after the numerical derivative. To provide a better estimation of the observables with the numerical derivatives, it is important to investigate the origin of the irregular behavior and to study alternatives.

Fig. 4.

|$T$| dependence of the relative residual of the internal energy, which is evaluated for |$D_{\mathrm{cut}}=12$|⁠, 16, 20 on a |$V=(16)^2$| lattice.

To understand the origin of this behavior, we write the right-hand side of Eq. (4) in the form

\begin{align} T^{(1)}_{IJ} = \sum_{m=1}^{D_{\mathrm{cut}}}\lambda_{m} {u}^{(m)}_I {v}^{(m)}_{J}, \label{approximated_tensor1} \end{align}

(8)

where |$\vec{u}^{(m)}$| (⁠|$\vec{v}^{(m)}$|⁠) is the left (right) singular vector corresponding to the |$m$|th singular value |$\lambda_{m}$|⁠. |$T^{(1)}_{IJ}$| is an approximation of the tensor |$T_{ijkl}$| for |$I=(i,j)$| and |$J=(k,l)$|⁠. We assume that a set of singular values and corresponding singular vectors |$(\lambda, \vec{u}(\lambda), \vec{v}(\lambda))$| smoothly change under a local variation of the parameters.

We now consider the case in which the level crossing takes place such that the |$K$|th and |$(K+1)$|th singular values are interchanged at some value of the parameter. The crossover is not important in the case of |$K \neq D_{\mathrm{cut}}$| because both the |$K$|th and |$(K+1)$|th singular values are included (or not included) in Eq. (8). In the case of |$K=D_{\mathrm{cut}}$|⁠, however, the crossover could make Eq. (8) change drastically: the |$D_{\mathrm{cut}}$|th singular vector before the crossover becomes the |$(D_{\mathrm{cut}}+1)$|th one after the crossover and vice versa, while the |$D_{\mathrm{cut}}$|th singular value changes continuously.

In Fig. 5 we trace the continuous move of the |$D_{\mathrm{cut}}$|th and |$(D_{\mathrm{cut}}+1)$|th singular values around |$T_{\rm ref}$| (gray band), where the |$D_{\mathrm{cut}}$|th singular value (open circle) and the |$(D_{\mathrm{cut}}+1)$|th singular value (solid circle) are interchanged. Those singular values are obtained in the course-grained tensor after six renormalization steps with |$D_{\mathrm{cut}}=12$|⁠. We should note that the |$(D_{\mathrm{cut}}+1)$|th singular value becomes the |$D_{\mathrm{cut}}$|th singular value after the level crossing. The purple line is the |$D_{\mathrm{cut}}$|th singular value included in Eq. (8), and the green dotted line is the |$(D_{\mathrm{cut}}+1)$|th one. This behavior suggests that the discontinuity of the result of the free energy does not come from the singular values in Eq. (8) but from the discontinuous change of the singular vectors.

Fig. 5.

An example of the crossover of the |$D_{\mathrm{cut}}$|th and |$(D_{\mathrm{cut}}+1)$|th singular values, where |$D_{\mathrm{cut}}=12$|⁠, as a function of |$T$|⁠. See text for a description.

Let us consider the following modification for the approximation of the tensor at the final step of SVD:

\begin{eqnarray} T^{(2)}_{IJ} = \left\{ \begin{array}{ll} T^{(1)}_{IJ} & \quad {\rm for} \ T < T_{\rm ref}, \\ \displaystyle \sum_{m=1}^{D_{\mathrm{cut}}-1}\lambda_{m} {u}^{(m)}_I {v}^{(m)}_{J} + \lambda_{D_{\mathrm{cut}}+1} {u}^{(D_{\mathrm{cut}}+1)}_I {v}^{(D_{\mathrm{cut}}+1)}_{J} & \quad {\rm for} \ T \ge T_{\rm ref}. \end{array} \right. \label{approximated_tensor2} \end{eqnarray}

(9)

The meaning of this approximation is obvious from the definition. |$T^{(2)}$| coincides with |$T^{(1)}$| before the level crossing. After the level crossing, however, |$T^{(2)}$| continues to keep the same sets |$(\lambda, \vec{u}(\lambda), \vec{v}(\lambda))$|⁠, unlike |$T^{(1)}$|⁠. If the irregular behavior is caused by the change of the associated singular vectors, it is expected that the jump at |$T_{\rm ref}$| should vanish with the use of |$T^{(2)}$|⁠.

Figure 6 shows the residues obtained from |$T^{(2)}$|⁠, which are drawn by the blue curve. They smoothly depend on the temperature, and the jump at |$T_{\rm ref}$| has gone. It is also instructive to check the smooth behavior of the green curve, which represents the results in the case that the |$(D_{\mathrm{cut}}+1)$|th set is used instead of the |$D_{\mathrm{cut}}$|th set for |$T < T_{\rm ref}$| (and |$T^{(1)}$| is used for |$T \ge T_{\rm ref}$|⁠). We thus conclude that the irregular behavior is caused by the level crossing of the |$D_{\mathrm{cut}}$|th and |$(D_{\mathrm{cut}}+1)$|th singular values. More specifically, the replacement of the |$D_{\mathrm{cut}}$|th singular vector at the crossover point yields the jump in the results.

Fig. 6.

|$T$| dependence of the relative residue of the free energy evaluated with |$D_{\mathrm{cut}}=12$| on a |$V=(16)^2$| lattice. The blue broken line and the blue and green curves (points) are the relative residues of the free energy obtained from |$T^{(1)}$|⁠, |$T^{(2)}$|⁠, and |$T^{(3)}$|⁠, respectively. |$T^{(1)}$| is defined by Eq. (8), and |$T^{(2)}$| is defined by Eq. (9). |$T^{(3)}$| is the case that the |$(D_{\mathrm{cut}}+1)$|th set is used instead of the |$D_{\mathrm{cut}}$|th set for |$T<T_\mathrm{ref}$| and |$T^{(1)}$| is used for |$T\ge T_{\mathrm{ref}}$|⁠.

The irregular parameter dependence of the results obtained by the TRG method is caused by the level crossing of the singular values across the truncation order. The standard TRG employs a sharp cutoff such that the |$D_{\mathrm{cut}}$| largest singular values and the associated vectors are included in the renormalization steps and the others are discarded. In the following, we test other cutoff scheme such as a smooth cutoff to tame the misbehavior.

In order to define another truncation scheme, we introduce a weight function |$w_m$| to approximate the tensor |$T_{ijkl}$|⁠:

\begin{align} T_{ijkl}\simeq\sum_{m=1}^{D_{\mathrm{cut}}} w_{m} U_{(ij)m}\lambda_{m}V^{\dagger}_{m(kl)}, \label{scut} \end{align}

(10)

where |$U$| and |$V$| are unitary matrices and |$\lambda_m$| are singular values sorted in descending order. Note that Eq. (4) is given by choosing |$w_m=1$| for |$m\le D_{\mathrm{cut}}$| as the weight function. It can be expected that the crossover effect depends on |$w_m$| and may become weaker if we employ a smoother cutoff function for |$w_m$|⁠. Note that the introduction of |$w_m$| itself does not demand extra computational cost.

As possible choices of cutoff schemes we consider two types of weight functions: (A) a “slanting cut” given by

\begin{eqnarray} w_m^{\rm (A)}= \begin{cases} 1&(1\leq m\leq D_{\mathrm{cut}} - \Delta) , \\ \frac{D_{\mathrm{cut}} - m}{\Delta} & (D_{\mathrm{cut}} - \Delta <m\leq D_{\mathrm{cut}}), \end{cases} \label{eq:smoothcut_a} \end{eqnarray}

(11)

and (B) an “FDF cut” inspired by by the Fermi distribution function

\begin{align} w^{\rm (B)}_{m} = \frac{1}{{e}^{(m-D_{\mathrm{cut}})/\sigma}+1}. \label{eq:smoothcut_b} \end{align}

(12)

Figure 7 shows examples of |$w_{m}^{\rm (A)}$| and |$w_{m}^{\rm (B)}$| in which we take |$D_{\mathrm{cut}}=12$|⁠. |$\Delta$| in |$w^{\rm (A)}$| and |$\sigma$| in |$w^{\rm (B)}$| are the tunable parameters which basically give the smeared size of the cutoff. Here we employ |$\Delta=3$| and |$\sigma=1.5$|⁠.

Fig. 7.

Weight factors |$w_{m}^{(\mathrm{A})}$| and |$w_{m}^{(\mathrm{B})}$|⁠.

Figure 8 shows the relative residues of |$F$| obtained by these two cutoffs, in which a smoother temperature dependence is obtained compared with Fig. 3. The FDF cut method (B) provides better behavior of the relative residual. Figure 9 shows the relative residues of |$U$| computed by the numerical derivative of |$F$|⁠. Even though there are small jumps, the relative residual of |$U$| has a smoother |$T$| dependence qualitatively compared with those evaluated by the sharp cutoff method as shown in Fig. 4. It is confirmed that the smooth cutoff scheme is effective in taming the irregular parameter dependence found in the sharp cutoff scheme in the standard TRG method.

Fig. 8.

|$T$| dependence of the relative residual of the free energy on a |$V=(16)^2$| lattice. The dotted line is for (A), the slanting cut method with |$\Delta=3$| in Eq. (11), and the chain line is for (B), the FDF cut method with |$\sigma=1.5$| in Eq. (12).

Fig. 9.

|$T$| dependence of the relative residual of the internal energy on a |$V=(16)^2$| lattice.

4. Summary and discussion

We have discussed the issue of the irregular parameter dependence observed in TRG results. We have investigated its origin using the two-dimensional Ising model and concluded that the irregular behavior is caused by the level crossing between the singular values in the sharp cutoff scheme with |$D_{\mathrm{cut}}$|⁠.

When the level crossing occurs between the |$D_{\mathrm{cut}}$|th and |$(D_{\mathrm{cut}}+1)$|th singular values, the |$D_{\mathrm{cut}}$|th singular vector is replaced by a completely different one across the crossover point, though the |$D_{\mathrm{cut}}$|th singular value changes continuously as a function of the parameter. Thus the constructed tensor drastically changes and yields a jump in the numerical result at the crossover point.

We have shown that a smooth cutoff improves the irregular behavior of the free energy in the two-dimensional Ising model. This implies that the irregular behavior does not have an underlying physical reason because it is mostly removed by changing the cutoff schemes. Further improvements would be important in obtaining precise results in more complicated lattice models or higher-dimensional models with tensor network schemes.

Similar behavior can be observed in any RG method which retains |$D$| local bases because it is triggered by the interchange of bases. So, in testing another cutoff scheme, as demonstrated in this paper, one can study whether it is a physical phenomenon or not in other RG methods.

Acknowledgements

We would like to thank Ken-Ichi Ishikawa for encouraging our study. The numerical simulations were performed on the INSAM (Institute for Nonlinear Sciences and Applied Mathematics) cluster system at Hiroshima University. This work was supported by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) as “Exploratory Challenge on Post-K computer” (Frontiers of Basic Science: Challenging the Limits), Grant-in-Aid for Japan Society for the Promotion of Science (JSPS) Research Fellow (No. 18J10663), JSPS KAKENHI Grant Numbers JP16K05328 and JP19K03853, and the CUniverse research promotion project of Chulalongkorn University (Grant CUAASC).

Funding

Open Access funding: SCOAP|$^3$|⁠.