Generative modeling, with its successes in image generation, text synthesis, and machine translation, presents a promising avenue for demonstrating quantum computational advantage. A key performance metric for generative models is generalization, the ability to generate new data similar to the training data. However, defining and evaluating generalization in generative modeling is complex. Existing frameworks often lack a clear, quantitative comparison between quantum and classical models in relevant real-world scenarios. This paper addresses this gap by introducing a rigorous framework to compare generalization performance, focusing on what the authors term 'potential PQA' (PPQA), where quantum models are compared against the best-known classical algorithms for a given task. The framework defines clear 'race conditions' for fair comparison, focusing on real-world constraints such as limited data and computational resources. The authors use a well-defined generative modeling task with 20 variables (qubits) to demonstrate their framework.

The authors acknowledge the lack of a concrete quantitative comparison between quantum generative models, particularly Quantum Circuit Born Machines (QCBMs), and classical state-of-the-art generative models in terms of generalization. Previous works have focused on computational quantum advantage in settings not directly relevant to real-world applications or using datasets that inherently favor quantum models. While some studies have proposed metrics for comparing classical and quantum models, they often do not address the crucial aspect of generalization in practical settings. This paper aims to fill this gap by providing a numerical comparison using a rigorous framework that focuses on generalization in realistic scenarios.

The proposed framework defines four types of PQA: provable PQA (PrPQA), robust PQA (RPQA), potential PQA (PPQA), and limited PQA (LPQA). The study focuses on PPQA, comparing quantum models against best-known classical algorithms. A sports analogy (hurdles race) illustrates the framework's concept of context-dependent evaluation. The 'race conditions' are pre-defined, involving limited data and computational resources, reflecting real-world constraints. The Evens distribution (a re-weighted version where bitstrings have an even number of ones) is used as a testbed, with a synthetic cost function (negative separation cost) that encourages the generation of low-cost samples. Five generative models are compared: QCBMs, RNNs, Transformers, VAEs, and WGANs. Two 'tracks' are defined for the comparison, reflecting different resource limitations: Track 1 (T1) limits the sampling budget but allows unlimited cost function evaluations, while Track 2 (T2) limits the number of unique, valid samples and cost function evaluations. Quality-based generalization is evaluated using three metrics: minimum value (MV), utility (U), and quality coverage (Cq). The models are trained with different training dataset sizes (epsilon = 0.01 and 0.001), and the metrics are averaged over multiple random seeds. The authors use Optuna for hyperparameter optimization and train all models for 1000 epochs.

The results show that QCBMs outperform other state-of-the-art classical generative models in the data-scarce regime (epsilon = 0.001), particularly in Track 1 (T1) which is characterized by a limited sampling budget. In T1, QCBMs achieve the lowest utility while maintaining competitive minimum value and quality coverage. In T2, which has limited cost function evaluation budget, QCBMs show competitive performance with VAEs and better quality coverage than WGANs, Transformers, and RNNs. Even with epsilon = 0.01 (more data), QCBMs show competitive performance, particularly in terms of utility. Importantly, the QCBMs used have fewer parameters than other models. The study also evaluates pre-generalization and validity-based generalization metrics (detailed in supplementary materials), finding that QCBMs offer a good balance between quality-based and validity-based generalization. The VAEs and WGANs tend to sacrifice the latter aspects in favor of quality-based generalization.

The findings demonstrate the potential of QCBMs in real-world scenarios with limited data. The efficiency of QCBMs in the scarce-data regime contrasts with the typically high data requirements of classical deep learning models. The framework presented provides a clear and quantitative way to assess PQA in generative modeling, moving beyond purely computational considerations. The use of the Evens distribution, while synthetic, provides a useful testbed with relevance to marginal probabilistic inference and constrained optimization problems. Future work could involve exploring other real-world datasets and incorporating additional constraints into the generative models.

This paper establishes a framework for comparing quantum and classical generative models based on quality-based generalization. The defined 'tracks' and metrics offer a rigorous approach for evaluating potential quantum advantage (PPQA) under realistic constraints. QCBMs demonstrated superior performance in data-scarce settings, highlighting their promise for real-world applications. Future research should expand the framework to include a broader range of generative models and datasets, and explore more varied 'race conditions'.

The study uses a synthetic dataset (Evens distribution), which may not fully capture the complexities of real-world data. The choice of cost function, while motivated, could also influence the results. Furthermore, only a limited set of classical generative models are considered. While the framework is intended to be general, the 'tracks' defined are not exhaustive, and other relevant constraints might be explored in the future. The hyperparameter search might also have been limited by the computational resources available to the authors. Finally, while the lower parameter count of QCBMs is noted, the absolute performance and complexity of the models are impacted by many factors.