Generalisable 3D printing error detection and correction via multi-head neural networks

Additive manufacturing (AM), particularly material extrusion, holds significant promise across various sectors (healthcare, aerospace, robotics) due to its cost-effectiveness, minimal post-processing, and material versatility. However, its susceptibility to diverse errors, ranging from minor dimensional inaccuracies to complete build failures, hinders widespread adoption in end-use applications, especially safety-critical ones. Current error detection methods, relying on human observation or specialized sensors (current, inertial, acoustic), are often insufficient for continuous monitoring, real-time correction, or generalizability across different parts, materials, and printing systems. Camera-based approaches offer potential due to cost-effectiveness and data richness, but existing single or multi-camera systems struggle with limited viewpoints, the obstruction of the print head, and the lack of generalizable algorithms for diverse errors. While machine learning shows promise, existing work lacks generalizability across various parts, often detects only a single error modality, and rarely incorporates real-time correction. To address these limitations, this research proposes a new approach that combines automated data acquisition, a powerful multi-head neural network, and a sophisticated feedback control loop to enable generalizable, real-time error detection and correction in material extrusion AM.

Existing research on monitoring extrusion AM has explored various sensor modalities. Current, inertial, and acoustic sensors have shown success in detecting large-scale errors, but their high cost and limited data richness restrict their application and real-time correction capabilities. Camera-based approaches, using single or multiple cameras, are more affordable and provide richer data. However, single cameras provide limited information, while multi-camera systems are more complex and expensive. Traditional computer vision techniques have achieved some success but lack generalizability across different parts, printers, materials, and setups. Recent applications of machine learning in extrusion AM error detection have shown promise, but typically only address single error modalities or specific parts, limiting their practical applicability. Furthermore, real-time correction mechanisms remain underdeveloped, typically lacking the speed and generalizability required for practical implementation.

This study introduces CAXTON, a system that automates data acquisition and labeling for training a deep learning model for real-time error detection and correction. The system utilizes a network of eight Creality CR-20 Pro 3D printers, each equipped with a low-cost webcam and a Raspberry Pi 4 for data collection and processing. A custom Python script manages the entire process, from STL file selection and slicing with randomly sampled parameters (scale, rotation, infill density, etc.) to toolpath generation, data acquisition (images captured every 0.4 seconds), and storage. Each image is automatically labelled with precise printing parameters (hotend and bed temperatures, flow rate, lateral speed, Z offset). The automation allows for the generation of a large and diverse dataset (initially 1,272,273 images, cleaned to 946,283) from 192 different parts, with parameter values sampled from uniform distributions to create varied printing conditions. Data augmentation techniques (rotation, perspective transform, cropping, flipping, colour jitter) are applied to enhance the dataset and improve the model's generalizability. A multi-head residual attention network, with a shared backbone and four separate output heads (one for each parameter: flow rate, lateral speed, Z offset, hotend temperature), is trained on this dataset. The network architecture uses attention modules to enhance robustness to noisy labels. The training process is divided into three stages: (1) training on a single-layer dataset, (2) transfer learning to the full dataset, and (3) fine-tuning on a balanced dataset to address class imbalances. For online correction, a feedback loop is implemented where images are captured, processed by the network, and parameter adjustments are sent to the printer based on the predicted parameters, using a proportional correction scheme and toolpath splitting for faster response times. The system includes a novel automated part removal system to facilitate continuous printing. Visualizations of the network’s predictions (guided backpropagation and Grad-CAM) are used to interpret how the network makes decisions and build trust in the system.

The trained multi-head neural network achieves high accuracy in simultaneously predicting the four key printing parameters. The system demonstrates effective real-time error correction across diverse 2D and 3D geometries, materials (PLA, TPU, ABS-X, PVA HT+, carbon fiber-filled), printers (Creality CR-20 Pro, Lulzbot Taz 6), toolpaths, and even extrusion methods (FDM, DIW). The automated correction successfully recovers prints with manually introduced errors, including multiple simultaneous parameter errors, and those initiated with poor parameter combinations. The system demonstrates automatic parameter discovery for new materials. The visualizations reveal that the network learns to focus on the most recently extruded material, facilitating rapid error correction. The system achieves an order of magnitude improvement in correction speed compared to existing work. The implementation successfully handles diverse materials including PDMS, mayonnaise and ketchup with different nozzle sizes. The analysis shows that similar features are used across parameters to identify under and over extrusion. The addition of multiple heads to the network even leads to an improvement in the prediction of individual parameters.

The results demonstrate the effectiveness of the proposed approach in achieving robust and generalizable real-time error detection and correction in extrusion-based 3D printing. The combination of automated data acquisition, a powerful multi-head neural network, and a sophisticated feedback control loop addresses the limitations of previous methods. The system's ability to handle diverse scenarios highlights its potential for practical application in industrial settings. The improved speed and accuracy of correction compared to existing work suggest a significant advancement in AM feedback control. The ability to perform parameter discovery and self-correct across different materials opens doors for more efficient and reliable AM processes.

This study presents a significant advancement in real-time error detection and correction for extrusion-based 3D printing. The combination of automated data collection, a powerful multi-head neural network, and a novel feedback control loop delivers highly accurate, fast, and generalizable results across diverse geometries, materials, and printing systems. Future research should focus on expanding the dataset, investigating the influence of parameter interaction on prediction accuracy, incorporating additional sensor modalities, and extending the system to handle more complex errors.

While the system demonstrates impressive generalizability, further testing with a wider range of printers and materials is needed to enhance robustness. The dataset, while large, exhibits some class imbalances, particularly for low Z offset values, which could affect the model's performance. The system does not address all error modalities (mechanical failures, electrical issues, large-scale defects like warping). Integrating global monitoring with the local nozzle-focused approach could improve the detection of larger-scale errors. The current random sampling of 3D models, slicing settings, and parameter values introduces some inherent bias which could be minimized with a more systematic and balanced dataset.