A framework for demonstrating practical quantum advantage: comparing quantum against classical generative models

The paper addresses how to rigorously assess and compare quantum and classical generative models with respect to generalization in realistic, application-driven settings, to search for practical quantum advantage (PQA). Generative modeling has broad impact in images, text, translation, and is a promising area to demonstrate quantum usefulness. However, defining generalization for generative models is challenging and often model-dependent. Building on a model-agnostic, sample-based framework for discrete spaces, the authors propose a quantitative race between quantum Circuit Born Machines (QCBMs) and state-of-the-art classical models (RNNs, Transformers, VAEs, WGANs), using real-world-motivated rules and metrics. They define a taxonomy of PQA (provable, robust, potential, limited) and focus on potential PQA (PPQA), meaning comparison against the best-known classical methods under clearly specified resource constraints. Using a sports (hurdles race) analogy, they emphasize that performance depends on the track (evaluation rules), data size, ground-truth distribution, and cost constraints. The study aims to unambiguously set competition conditions to evaluate generalization quality, with a focus on scarce data regimes and optimization-relevant metrics.

The authors situate their work among prior efforts on quantum advantage and evaluation of generative models. Earlier demonstrations emphasized computational quantum advantage or artificial settings not aligned with practical use. Some works proposed quantum utility indices or metrics for variational calculations, and others examined training quality comparisons without addressing generalization. A quantum-inspired advantage was shown for tensor network Born machines over GANs in certain settings. Prior work also introduced a generalization framework for discrete spaces and examined QCBMs’ ability to generalize. However, a broad, concrete quantitative comparison between QCBMs and diverse state-of-the-art classical generative models on generalization has been lacking. The paper extends beyond limited PQA (comparing quantumized versions of classical models) to potential PQA by comparing against multiple strong classical models under application-driven constraints and metrics.

Models compared: Quantum Circuit Born Machines (QCBMs; trained with gradient-free optimization) and classical Recurrent Neural Networks (RNNs), Transformers (TFs), Variational Autoencoders (VAEs), and Wasserstein GANs (WGANs). Hyperparameters for all models were tuned via Optuna grid search, selecting configurations that achieved the lowest utility after 100 training steps. All models were trained for N_epochs = 1000, with metrics evaluated every 100 steps. Results are averaged over N_seeds = 10 random initializations; error bars are one standard deviation. Dataset and task: A 20-variable binary generative modeling task using a re-weighted Evens (parity) distribution where only bitstrings with an even number of ones have non-zero probability. A synthetic cost function, the negative separation cost, is defined as c(x) = −(z + 1), where z is the longest run of consecutive zeros in bitstring x (e.g., c('11100011') = −4). The known global optimum has cost −(N_var − 1) for the bitstring '100...001'. Training datasets are constructed for two data fractions ε ∈ {0.01, 0.001}, chosen so that both have the same minimum observed cost (−12) to control for minimum-cost effects. Re-weighting: Training probabilities are re-weighted to emphasize low-cost samples: P_train(x) = exp(−β c(x)) / Σ_{y∈D_train} exp(−β c(y)), with β = β′/2, and β′ the standard deviation of costs in D_train; P_train(x) = 0 if x ∉ D_train. Re-weighting is applied consistently across models to encourage generation of lower-cost unseen samples while avoiding overfitting (i.e., preventing KL divergence from vanishing). Tracks (evaluation rules): Two realistic, application-driven tracks constrain evaluation resources. - Track 1 (T1): Fixed total query budget Q. Here Q = 10^6 samples from the model are used to compute quality metrics. This setting reflects limited sampling budgets (e.g., costly sampling on real quantum hardware) but inexpensive cost evaluation. - Track 2 (T2): Fixed budget of unique, unseen, valid samples Q_u. Sampling continues until Q_u = 10^2 unique, unseen, valid samples are collected (or as many as can be obtained if the model collapses). This models scenarios with expensive cost evaluation (e.g., molecule design, drug discovery) where repeated evaluations are to be avoided. Metrics (quality-based generalization): Evaluated on the set of generated samples G and the out-of-bag unique, unseen, valid subset G_oob. - Minimum Value (MV): MV = min_{x∈G} c(x), the lowest cost among unseen, valid outputs. - Utility (U): Average cost over the best 5% (lowest cost) of G_oob, reducing sensitivity to single extreme samples. - Quality Coverage (C_q): Fraction of unique, unseen, valid samples with cost lower than the minimal cost in the training set: C_q = |{x ∈ G_oob : c(x) < min_{y∈D_train} c(y)}| / Q (for T1) or normalized by Q_u (for T2). Training/evaluation protocol: For each ε, the same training data are used across models. All models trained for 1000 steps, metrics aggregated every 100 steps. The sampling cost for metric evaluation is not counted against training budgets to focus on generalization performance rather than training efficiency. Simulations were performed using Orquestra; hyperparameters and parameter counts are provided in supplementary materials.

- General observations: For both ε = 0.01 and ε = 0.001, all models generated unseen samples with costs lower than the minimum observed training cost (−12), indicating successful quality-based generalization. ε = 0.01 - Track 1 (T1, Q = 10^6): Minimum Value (MV) drops rapidly within the first 100 steps for all models. VAEs, WGANs, and QCBMs converge to MV = −19 (best observed), while RNNs and TFs degrade (MV increases) with additional training due to overfitting. Utility (U): VAE consistently achieves the lowest U, followed by QCBM, then other models. QCBM shows a monotonic decrease in U and competitive C_q, indicating both improving quality and diversity of high-quality solutions. - Track 2 (T2, Q_u = 10^2): WGAN achieves the best performance across C_q, U, and MV. ε = 0.001 (scarce-data regime) - Track 1 (T1): QCBM attains the lowest Utility (U) among all models, while maintaining competitive MV and C_q. This highlights QCBM’s strength under limited data with constrained sampling budgets. - Track 2 (T2): QCBM is competitive with VAE in MV and U and yields the best Quality Coverage (C_q), outperforming WGAN, TF, and RNN. Overall - QCBMs demonstrate the strongest quality-based generalization in the scarce-data regime, particularly under constrained sampling (e.g., ε = 0.001, T1), with competitive MV and C_q and superior trends in U. Despite having the fewest parameters (per supplementary materials), QCBMs match or outperform classical models with 10–100× more parameters. QCBMs also provide a better balance between quality-based and validity-based generalization compared to VAEs and WGANs, which tend to sacrifice pre-/validity-based aspects. These results constitute promising evidence toward potential PQA for quantum generative models in data-limited settings.

The study formalizes a practical, application-driven framework for comparing quantum and classical generative models toward potential PQA by specifying evaluation tracks that reflect real-world constraints. The results show that QCBMs can generalize effectively in scarce-data scenarios, generating low-cost, diverse, and unseen samples while maintaining stable utility improvements. This is particularly relevant when generative modeling is coupled to optimization tasks where high-quality (low-cost) and diverse solutions are valuable. Under T1 (limited sampling budget) and small ε, QCBMs’ superior or competitive performance indicates their potential utility when sampling resources are constrained (as with near-term quantum hardware). Under T2 (limited unique evaluations), QCBM remains competitive, especially in quality coverage, suggesting robustness in scenarios where cost evaluations are expensive and repeated samples should be minimized. By focusing on quality-based generalization rather than raw speed, the work emphasizes practical advantage measures aligned with real-world application needs, contributing a concrete step toward establishing PPQA for generative models.

The paper defines four classes of practical quantum advantage (provable, robust, potential, limited) and operationalizes potential PQA (PPQA) via a quantitative race among quantum and state-of-the-art classical generative models using real-world-motivated tracks and metrics. Using a 20-variable parity-constrained task with a synthetic cost, and evaluating quality-based generalization via minimum value, utility, and quality coverage under two resource-constrained tracks, the study finds that QCBMs are highly competitive and often superior in scarce-data regimes, especially with limited sampling budgets, while maintaining good diversity of high-quality solutions. The framework provides a clear, reproducible procedure for establishing practical criteria for advantage in generative modeling with implications for combinatorial optimization. Future work should expand to additional quantum and classical architectures (e.g., diffusion, flows), incorporate constraint-embedded models (e.g., U(1)-symmetric architectures), include more realistic datasets and training budget accounting, and explore synergistic classical–quantum approaches (e.g., tensor-network-aided circuit pretraining) to further assess and enhance the prospects for PQA.

- Evaluation tracks are illustrative and not exhaustive; other rules (e.g., updating training data per step as in GEO, accounting for training sampling costs) could alter outcomes. - Synthetic dataset (parity/Evens) with a synthetic cost function may not capture complexities of real-world domains; generalization to larger, realistic datasets remains to be demonstrated. - System size limited to 20 variables; although simulable efficiently, scalability to larger problem sizes and execution on real quantum hardware require further study. - Focus on quality-based generalization; pre- and validity-based metrics are discussed in supplementary materials but not central to the main comparisons. - Hyperparameter selection prioritized utility after 100 steps; alternative selection criteria (e.g., MV, C_q, or combinations) could yield different model rankings. - Training efficiency and resource usage were not constrained (unlimited training assumed); results focus on evaluation-stage resource constraints only. - Limited set of classical and quantum models; broader inclusion (e.g., diffusion models, normalizing flows, additional quantum generative models) could refine conclusions. - Findings demonstrate potential PQA rather than robust or provable PQA; advantages may be sensitive to dataset, metrics, or future improvements in classical algorithms.