Exploring Innovative Approaches to Synthetic Tabular Data Generation

The increasing demand for large datasets in machine learning faces challenges in data acquisition, especially in sensitive areas like healthcare and cybersecurity. Synthetic data generation for tabular data offers a potential solution. This review differs from previous work by providing a comprehensive classification of algorithms and generation mechanisms, along with a detailed discussion of evaluation metrics. It aims to bridge gaps in the literature and provide valuable insights for researchers and practitioners. The study acknowledges the rise of Federated Learning (FL) and federated synthesis for privacy-preserving data generation, but notes the need for further research on privacy risks. The review also highlights the challenges of unbalanced datasets and the crucial need for privacy preservation, particularly in healthcare, where the protection of sensitive patient information is paramount. The review uses BERTopic, a topic modeling library leveraging BERT embeddings and HDBSCAN clustering, to analyze the collected studies, identifying key themes and implications for synthetic data generation.

The literature review explores existing research and methodologies in synthetic data generation across various domains (healthcare, retail), encompassing numerical, categorical, and temporal data. It examines statistical-based generation methods like GenerativeMTD and the divide-and-conquer (DC) strategy, highlighting their strengths in preserving complex data relationships and maintaining data balance. Machine learning-based generation, including cGANs, TGANs, and VAEs, is also reviewed, showcasing their use in improving prediction accuracy in healthcare, agriculture, and software engineering. The review discusses GANBLR, a novel model that enhances both performance and interpretability. It also examines privacy preservation techniques, such as Duo-GAN and methods using GANs for generating private tabular data, highlighting their application in cybersecurity and healthcare. Finally, it analyzes evaluation metrics, including fidelity metrics (like TabSynDex) and utility metrics, assessing their role in evaluating synthetic data quality.

The authors employed BERTopic, a topic modeling technique utilizing BERT embeddings and HDBSCAN clustering, to analyze the literature. Preprocessing steps, including tokenization, lowercasing, stop-word removal, and lemmatization, were applied to ensure data cleanliness. BERTopic dynamically adjusted to the data complexity without requiring a predefined number of topics, preserving significant words in topic descriptions. The resulting topics were visualized in bar charts displaying c-TF-IDF scores, quantifying term relevance based on frequency within a topic and uniqueness across the corpus. The methodology allowed for a nuanced analysis of the literature, providing insights into key themes and implications for synthetic data generation.

The review categorized 70 data generation algorithms and 6 production mechanisms. Statistical methods like GenerativeMTD and the DC strategy showed promise in preserving data relationships and maintaining balance, outperforming other methods in some cases. Machine learning approaches, particularly GANs and VAEs, improved prediction accuracy across various domains, including healthcare (fluid overload prediction, cancer diagnosis), agriculture (crop classification), and software engineering (effort estimation). GANBLR emerged as a superior model, improving performance and interpretability in tabular data generation. Privacy-preserving techniques, including Duo-GAN, showed promise in generating realistic data while protecting user privacy. Evaluation metrics, such as TabSynDex, were discussed for assessing synthetic data quality. The application of synthetic data generation is highlighted across various domains including healthcare (improving medical diagnoses, prognoses, and addressing class imbalances in datasets), and retail (addressing data scarcity in prediction models).

The review demonstrates the significant advancements and innovative approaches in synthetic data generation, particularly addressing data scarcity, privacy, and class imbalance. Statistical and machine learning-based methods offer practical solutions, but their transferability across domains requires further investigation. GAN-based approaches show promise in healthcare, but further research is needed to identify the most effective model. Privacy preservation remains a crucial concern, requiring robust techniques like Duo-GAN to protect sensitive information. Standardized evaluation metrics like TabSynDex enable better comparison of different generation methods. The applications in healthcare and retail highlight the practical impact of synthetic data generation, though challenges in data quality, computational resources, and interpretability need to be addressed.

This review provides a comprehensive overview of innovative approaches to synthetic tabular data generation. Both statistical and machine learning-based methods have shown promise, but challenges remain in areas such as ensuring data fidelity, preserving privacy, and balancing computational efficiency with model interpretability. Future research should focus on developing more sophisticated generative models, integrating domain expertise, establishing robust evaluation methodologies, and addressing ethical implications to ensure responsible data use.

While this review provides a comprehensive overview of current research, it acknowledges that the rapidly evolving nature of the field may limit the scope of some discussions. The focus primarily on tabular data may not fully represent the diversity of synthetic data generation techniques. Furthermore, the evaluation of different methods might be limited by the availability of standardized datasets and evaluation metrics.