Training neural networks is computationally expensive, raising economic and ecological concerns. Alternative computing systems, such as analog Ising machines, are being explored to accelerate this process. Analog Ising machines map computational problems to the Ising model, leveraging the system's natural tendency to find low-energy configurations. While various neural networks can be implemented, current training methods are inefficient due to the need for Boltzmann sampling, which estimates neuron activation probabilities in thermal equilibrium. Analog Ising machines operate at low temperatures, preventing Boltzmann sampling at arbitrary temperatures. Existing alternative sampling methods suffer from inaccuracy, complex temperature control, and performance overhead. This research proposes a method to overcome this limitation and enable efficient Boltzmann sampling with analog Ising machines.

The paper reviews existing research on analog Ising machines, highlighting their successes in combinatorial optimization but also their limitations in performing Boltzmann sampling for machine learning applications. It discusses previous attempts at Boltzmann sampling using analog Ising machines and their shortcomings, emphasizing the need for a more efficient and accurate approach. The authors acknowledge the computational cost of Boltzmann sampling in training neural networks and its significance in various other fields such as drug research and finance.

The authors propose a novel method for Boltzmann sampling using analog Ising machines by injecting broadband noise from an analog noise source. This noise acts as a randomizing element, preventing convergence to a stable state and driving the system towards thermal equilibrium at a temperature determined by the noise strength. The research uses a time-multiplexed opto-electronic Ising machine for experimental demonstration. This machine is a hybrid system comprising an opto-electronic analog nonlinear system for generating spin states and an FPGA for spin coupling. Gaussian white noise is injected into the system, and samples are collected after each iteration. The accuracy of the sampling is quantified using the Kullback-Leibler divergence, comparing the sampled distribution to that obtained using the Metropolis-Hastings algorithm. The relationship between noise strength and temperature is investigated, establishing a linear relationship for controlling the temperature a priori. The method is then applied to unsupervised training of neural networks, specifically restricted Boltzmann machines (RBMs), using the Ising machine for Boltzmann sampling to replace software-based methods. Numerical simulations of a spatially multiplexed analog Ising machine are used to assess scalability and speed for larger problem sizes, comparing the results to MCMC-based sampling. The autocorrelation function of the Ising energy is used to estimate the sampling rate.

The experimental results demonstrate that noise-induced sampling accurately approximates Boltzmann distributions. The Kullback-Leibler divergence between the Ising machine-generated distribution and the MCMC-generated distribution is low (e.g., D_KL = 0.04 for a specific example), indicating a high degree of accuracy. The noise-induced sampling method significantly outperforms discontinuous sampling methods in both accuracy and speed. A linear relationship is established between noise variance and temperature, simplifying temperature control. Unsupervised training of RBMs for handwritten digit recognition achieves accuracy comparable to MCMC-based training, with the Ising machine even showing slightly improved performance in some metrics. Simulations of spatially multiplexed Ising machines demonstrate the scalability of the method to large-scale problems (N ≤ 8192) with maintained accuracy. The simulated sampling rate for a spatially multiplexed Ising machine reaches several gigasamples per second (GSamples/s), several orders of magnitude faster than software-based methods. Even software simulations of the noise-induced sampling method outperform the Metropolis-Hastings algorithm in terms of runtime and number of iterations for large problems. The authors explore the universality of their method by testing it on different types of analog Ising machine models (polynomial, clipped, and sigmoid nonlinearities) demonstrating its applicability across diverse implementations.

The findings demonstrate that noise-induced sampling offers a significant advantage over existing methods for Boltzmann sampling with analog Ising machines. The increased speed and accuracy of this method bridge the efficiency gap in using analog Ising machines for machine learning. The linear relationship between temperature and noise simplifies temperature control and enables a priori setting of temperature. The improved sampling enables accurate and efficient training of more complex neural network architectures. The universality of the method suggests broad applicability across diverse analog Ising machine implementations. The order-of-magnitude speedup in sampling holds significant potential for various applications, including drug research and finance.

Noise-induced sampling provides a substantial improvement in both speed and ease of use for Boltzmann sampling with analog Ising machines, achieving orders of magnitude speedup compared to software methods. This technique is applicable to various analog Ising machine designs and demonstrates accuracy comparable to or exceeding software-based methods. This significantly advances the utility of analog Ising machines for machine learning and opens doors for wider applications beyond combinatorial optimization. Future research could focus on further optimizing the analog Ising machine architectures and exploring additional applications in diverse fields.

While the proposed method shows promising results, limitations include potential mapping errors for smaller problem sizes where the system is more likely to be in the ground state. The accuracy of the Ising machine-based sampling can be affected by temperature, especially at low temperatures where trapping in local energy minima becomes more prevalent. The study primarily focuses on RBMs; further investigation is needed to assess the performance on more complex neural network architectures. The simulation-based scalability analysis relies on a numerical model, and experimental verification for extremely large-scale problems is required.