The increasing complexity and size of machine learning models pose challenges for deployment on resource-constrained hardware. Traditional approaches prioritize model training performance and then optimize for memory and area footprint during deployment. This paper addresses this challenge by focusing on the direct generation of hardware-efficient classification circuits. The authors propose a methodology, termed "auto tiny classifiers", that automatically generates these circuits directly from tabular data, bypassing the intermediate step of training a conventional machine learning model and subsequent hardware translation. This approach is particularly relevant for tabular data, which often lacks the strong spatial or semantic relationships that deep neural networks excel at capturing in image or text data. Tabular data are ubiquitous across various applications, frequently found in resource-limited scenarios, making them ideal for low-power machine learning (tinyML) approaches. The authors argue that their approach offers a novel alternative to existing methods by leveraging the favorable properties of decision trees and mitigating the local minima problems associated with gradient-based methods. This allows for comparable prediction performance to state-of-the-art machine learning classifiers while drastically reducing hardware requirements.

The paper reviews existing approaches to machine learning hardware acceleration. It highlights the common practice of separating model training and hardware design optimization. The authors contrast traditional AutoML, Neural Architecture Search (NAS), and Neural Architecture and Implementation Search (NAIS) approaches with their proposed methodology. AutoML and NAS optimize for prediction accuracy but require manual translation to hardware description languages, a time-consuming and error-prone process. NAIS co-designs the neural network and hardware accelerator, still relying on pre-defined hardware pools. In contrast, the "auto tiny classifiers" approach uses an evolutionary algorithm to directly search the circuit space, generating a combinational circuit that is automatically translated into a synthesizable hardware description language. The paper also examines existing supervised classification methods for tabular data, such as gradient-boosted decision trees (XGBoost, CatBoost) and deep learning architectures (TabNet). Recent research suggests that optimized multilayer perceptrons (MLPs) can rival the performance of these methods, prompting the authors to use these as baselines for comparison.

The core of the methodology is an evolutionary algorithm based on evolving graphs by graph programming (EGGP). EGGP employs a 1+λ evolutionary technique, mimicking neutral drift of DNA. The algorithm iteratively generates and evaluates candidate circuits represented as graphs of logic gates. These graphs consist of input nodes, function nodes (representing logic gates), and output nodes. The algorithm uses mutation operators to modify the function nodes (changing the type of gate) and edges (changing the connections between gates). The fitness of a circuit is determined by its balanced accuracy on a training dataset. The algorithm runs until the validation set accuracy plateaus, or a maximum generation count is reached. The best-performing circuit is then automatically translated into register-transfer level (RTL) Verilog code, ready for hardware synthesis. The hyperparameters of the algorithm, such as the number of children per generation (λ), the mutation rate (p), and the target gate count, are user-defined. The input data is encoded based on user preferences (binary, one-hot, gray encoding) and the number of bits per input. The generated Verilog code is synthesized using a commercial tool, generating area, power, and timing reports. The full chip implementation involves additional steps, such as floor planning, place and route, ultimately producing a GDSII file for fabrication. The design incorporates input and output buffers to minimize data transfer overhead.

The authors evaluated their method across 33 tabular datasets from OpenML, UCI, and Kaggle. They compared their tiny classifiers against several state-of-the-art machine learning methods, including Google's TabNet, AutoGluon (using XGBoost, TabNeuralNet, and NNFastAITab), and optimized MLPs (both best-performing and smallest configurations). Across all datasets, the average accuracy of the tiny classifiers (78%) was second only to AutoGluon XGBoost (81%). A tenfold cross-validation study confirmed the robustness of the tiny classifiers, showing low variance in accuracy compared to XGBoost. Hardware synthesis results showed significant improvements in area and power consumption. Tiny classifiers consumed 0.04-0.97 mW and occupied 11-426 NAND2-equivalent gates, while the MLP baselines consumed 34-38 mW (86-118 times higher) and had significantly larger area. XGBoost also consumed more power and occupied significantly more area. The implementation on flexible substrates (using Pragmatic's 0.8 µm FlexIC) further demonstrated the advantages of tiny classifiers, which were 10-75 times smaller, consumed significantly less power, and had a six times higher yield than XGBoost. Notably, the area variation of tiny classifiers between binary and multiclass problems was minimal, contrasting with the significantly larger area increase observed in XGBoost.

The findings demonstrate that the proposed methodology successfully generates accurate and hardware-efficient classification circuits for tabular data. The comparable prediction accuracy to state-of-the-art machine learning techniques while achieving significant reductions in area and power consumption highlights the potential of this approach for resource-constrained applications. The superior performance and yield on flexible substrates further expand the applicability of this method to low-cost, near-sensor computing and smart packaging scenarios. The robustness of the tiny classifiers, as evidenced by the low variance in accuracy across cross-validation, adds to their practical value. The simplicity and directness of the approach, avoiding the intermediate steps of training a conventional model and translating it to hardware, makes it a promising direction for future research in efficient machine learning.

This paper presents a novel methodology for automatically generating hardware-efficient classification circuits for tabular data. The "auto tiny classifiers" approach leverages graph-based genetic programming to create small, low-power circuits that achieve comparable accuracy to state-of-the-art machine learning techniques. The results demonstrate significant improvements in area, power, and yield, particularly when implemented on flexible substrates. Future research could explore the application of this methodology to other data types and the integration of tiny classifiers into more complex systems.

The current implementation of auto tiny classifiers focuses on combinational circuits. Incorporating sequential logic might further enhance the capabilities and applicability of the generated circuits. The hyperparameter tuning might require experimentation to find optimal settings for specific datasets. Further exploration of different logic gate sets and encoding strategies could potentially improve performance and resource utilization. The current study focuses on classification tasks; extension to regression tasks could broaden its applicability.