A Machine Learning Approach to Duality in Statistical Physics

Stronger evidence of approximate duality

We thank the reviewer for the suggestion and agree that additional evidence strengthens our claims. We now provide such evidence, showing that a broad set of moments—including correlation length—is accurately matched across approximate duals. This close agreement suggests that the essential physics is preserved, a novel result to our knowledge.

To assess generalization, we compare feature statistics between approximate duals (both found from our experiments in the paper and from the hypothesized line $\beta + \kappa = const$) and the corresponding theoretic duals. Importantly,

• We evaluate various features not included in the training loss
• We compute these features on larger $24 \times 24$ lattices, beyond the training regime
• We include approximate duals along the hypothesized line for comparison

Due to computational constraints, training directly on large lattices like $24 \times 24$ is infeasible. Instead, we apply the learned mappings $G_{\theta}$ from $8\times8$ lattices to estimate features on larger systems without retraining.

In all the plots, the top panel shows average feature values across all approximate duals for $\beta_0 \in \\{0.2, 0.25, 0.3\\}$ —the original-frame $\beta$ values that yielded these approximate duals, and the lower panels show individual approximate duals (marked by x). Squares mark the features corresponding to theoretical dual configurations.

We consider three categories of features. For each category, we present two plots: (Framework) one based on approximate duals found by our framework, and (Hypothesized) another based on duals inferred from the hypothesized line $\beta + \kappa = const$, where the constant is chosen to intersect the known dual point $\beta_{\text{dual}}$. All links point to an anonymous repository hosting the plots.

• Product of consecutive links in a linear chain in a lattice of size 24x24: There are 24 such features. (not used in the training loss)

  • Framework (https://bit.ly/4hQnMJH)
  • Hyopthesized (https://bit.ly/4jhTOj7)
  • Both sets of approximate duals closely match theoretical expectations

• 13 features constructed from link products used in the training loss:

  • Framework (https://bit.ly/4lq2vtw)
  • Hyopthesized (https://bit.ly/41TzC16)
  • These features match well across both sets of approximate duals, despite being trained on smaller 8x8 lattices.

• 101 Features constructed from all possible (up to gauge equivalence) link products in a $3\times3$ grid: (not used in the training loss)

  • Framework (https://bit.ly/4lt5Tnx)
  • Hyopthesized (https://bit.ly/42rEtGM)
  • Even this exhaustive set of features shows strong alignment with the theoretical dual, reinforcing the robustness of our approach.

Regarding Figure 6, we appreciate the suggestion and will revise the caption to provide a clearer explanation, helping readers better interpret the figure.

Simplicity of ML methods

Although the ML methods are indeed simple, they certainly involve new important ideas; we did not find anywhere in the literature the task of learning both parameters in a Hamiltonian and an observable. We also thought that the application of these methods to statistical physics models was sufficiently broad, since it is one of the major branches of theoretical physics.

We could however have further emphasised the broader applicability of these methods; one such is Hamiltonian truncation, where the task is to compute the spectrum of some truncated hamiltonian to gain insights into the behaviour of the physical systems; it is routinely applied to the study of QFT phenomena, such as duality. RG flows, etc. Our methods could for instance be used in this context to discover accidental dualities like Chang duality (https://journals.aps.org/prd/abstract/10.1103/PhysRevD.93.065014), which have so far received no systematic explanation. More precisely, the task in this context would be to fit parameters in a truncated hamiltonian such that the distance between the eigenvalues of the hamiltonian, or the eigenvalues of some related observable of different models are minimised. This is precisely the type of task discussed in this paper.

Deviation from the hypothesized $\beta + \kappa$ line

We believe this happens because the idea that the physics is completely determined by $\beta + \kappa$ (as explained on p6, where this determines the single spin-flip probability) is an approximation; in practice there is always a finite probability for two spins to flip together, and in this case the observables will depend on \beta and \kappa individually. We believe this is the reason for systematic movement off of the $\beta + \kappa$ line.

We stress there is also likely a statistical reason for the movement off the line; not all runs find equally good minima of the loss due to random initializations etc. There is scope for improvement of this through engineering improvements.

Code Availability & Moments & Reproducibility

We would like to kindly remind the referee that a full working code was provided in the supplementary material. Feature computation is handled by src.utils_ising.generate_masks (which generates 13 link product masks) and src.utils_ising.feature_samplewise. We will also open-source the code and experimentation scripts to ensure full reproducibility.

Training Features We use a specific set of link product moments in the training loss. These are visually illustrated in this plot (https://anonymous.4open.science/r/temp_rebuttal_repository-0225/images/features.png), where the red highlights indicate the link products averaged across the lattice and samples to form statistical features.

How is our work different from Betzler,Krippendorf ?

Betzler-Krippendorf has a somewhat different focus, which includes an investigation of the advantage of a variety of different known dual representations. The part of this paper with the most overlap with ours is Section 3.3. The key differences are the following: (1) the input into the duality mapping is spin configurations sampled in the original frame, after which a second step of sampling is done: this is not the usual setup for duality in physics, where one usually just samples once (importantly, in a dual frame) and then performs a deterministic mapping, as in our work. (2) It seems that the loss function used in that work cannot be formulated unless the duality mapping of temperatures is known already, and thus that this work cannot be used to find new dualities, which our formalism in principle allows. We will include this discussion in the updated version of our manuscript.

Practical use case of our work

We have a somewhat broader perspective on duality transformations: we do not think that they are used only to compute n-point functions. Rather they provide an important perspective on the structure of the underlying physics. In particular, Kramers-Wannier duality is important not for calculations because it is the simplest example of an order/disorder duality. The discovery of a genuinely new duality – based on novel principles – would have a huge impact on physics in general.

In addition to the above orientational remarks, we would like to provide two more concrete directions that we hope to explore with this technology on a short time scale.

1. Much recent work (summarized in https://arxiv.org/abs/2308.00747) has focused on points in parameter space which are self-dual, in which there is a new symmetry of an exotic “non-invertible” type. Little is known on how these symmetries work in more general theories (such as the plaquette Ising model) – for example we do not know whether the symmetry exists on the phase-transition line away from \kappa = 0. Understanding whether or not a self-duality can exist at all away from \kappa = 0 is an important first step in understanding the symmetry.

2. There exist models with interesting phase transitions (e.g. the simple case of bond percolation, reviewed in https://arxiv.org/abs/math-ph/0103018) that are not described by any known local model. It is very interesting to ask whether there can be a (perhaps complicated) local model that describes them.

In both of these cases, we anticipate learning about deeper questions from the existence of a duality. In our eyes this is the true value of our work.

Do Dualities Reduce to Fourier Transforms?

In our opinion, many dualities go beyond Fourier transformations (a well-known example is AdS/CFT, which has no simple Fourier interpretation at all). As described above, our hope indeed is that other kinds of dualities – based on fundamentally new ideas – could be unveiled using these methods. One concrete example is Chang duality in \phi^4 theory (https://journals.aps.org/prd/abstract/10.1103/PhysRevD.93.065014). As we comment in the rebuttal to referee 3, our methods could realistically be applied to the discovery of such dualities, which are under intense investigation in the hep-th community.

Let us also note that our approximate dualities are already quite interesting and novel!

Sampling in the strong coupling regime

For these models the difficulty of sampling using MCMC in both the strong and weakly coupled regime is basically the same. (Difficulties arise close to the critical point where critical-slowing down causes familiar but surmountable problems). Obtaining analytic results (e.g. Feynman diagrams etc.) is of course hard at strong coupling, but – very importantly – our formalism does not require this at all.

Phase Dependence of the Method’s Performance

For statistical physics model this strong-weak paradigm is not really relevant; we should stress that it is equally easy to sample in both phases. The reasons why our methods work best in one phase seem more subtle and related to well-known difficulties encountered in the inverse Ising problem.

We thank the referee for their careful reading and interesting questions. We believe each of their three questions are interesting starting points for future research, and our more precise responses are below:

1. How would the method work for systems with continuous symmetries where there are many more possible dualities? In principle, the philosophy of the method – matching moments of an appropriate subset of observables – would still work for a system with continuous symmetries (e.g. we see no real reason why we could not obtain particle-vortex duality in 3d systems with a continuous U(1) symmetry, etc.). In practice, it seems conceivable that computationally this would be more onerous, as it would take longer to equilibrate the system when doing MCMC sampling (generally there would be gapless modes associated with Goldstone modes of the continuous symmetry which take longer to thermalize).

2. Is it possible to adapt this to quantum systems by working with transfer matrices? We think that it should be possible to use a similar approach for quantum systems without a sign problem: we would use the usual Trotter approach to reduce the quantum problem to an effective classical problem that could be simulated using very similar techniques. The problem would now be strongly anisotropic (as the time direction is now quite different from the spatial ones) and some thought should be put into the correct form of the neural network that maps observables (presumably it would also be strongly anisotropic) but in our opinion this would be manageable.

3. Is it worth exploring other neural networks that might better capture long-range correlations? Yes: making the neural network for the mapping have a larger spatial extent (i.e. going to “n” hops away on the lattice where n is a small number) is something that we are currently actively investigating as indeed some systems might require the need to capture longer-range correlations. Training becomes more difficult as this mapping is made more and more non-local. In principle there are known examples of duality (AdS/CFT being a particularly dramatic example) where the mapping is completely non-local; we do not think that our approach will reliably be able to find such examples as the space of mappings is simply too large.

---

Finally, in response to the question about what the approximate duals reveal about the ordered phase:

The approximate duals essentially describe a regime where the physics can be approximated by isolated single-spin flip events; this explanation shows that many different models are expected to flow to this regime, giving an example of the universality of RG (and conversely highlighting the difficulties in pinpointing the original Hamiltonian from data in such a regime)