This paper explores the use of transformer models as wavefunction ansatz for variational ground state searches in simulating qubit systems, specifically two-dimensional Rydberg atom arrays. The authors demonstrate that transformers achieve higher accuracies than conventional recurrent neural networks (RNNs) and introduce a novel architecture, large patched transformers (LPTFs), to accelerate simulations. LPTFs combine a powerful patched transformer model with an efficient patched RNN, significantly reducing computational costs while maintaining high accuracy. The results surpass state-of-the-art quantum Monte Carlo methods, enabling the study of large Rydberg systems and their phase transitions.

Kyle Sprague, Stefanie Czischek