Automated material synthesis using robotics and AI significantly improves material development efficiency. However, the handling of corrosive or flammable chemicals without human supervision poses substantial safety risks. Machine control errors can lead to accidents causing significant property damage and potential harm. To address these concerns and promote wider adoption of automated synthesis, safety mechanisms are crucial. Wet chemical synthesis frequently involves moving transparent vessels (flasks, beakers, vials). Incorrect placement can cause hazards during subsequent operations like stirring. While features like pressure sensing in robotic arms help mitigate these risks, they don't eliminate them entirely due to unpredictable external factors. Therefore, accurate detection of transparent vessel positions is vital for improving safety. Deep learning-based computer vision offers a powerful solution. Although computer vision is used extensively in various fields (autonomous vehicles, disease diagnosis, rehabilitation), object detectors specifically for automated material synthesis systems are lacking. This research aims to develop a high-performance object detector to identify the positions of transparent chemical vessels, enhancing safety in automated chemical synthesis laboratories.

Existing object detection models like You Only Look Once (YOLO) and Single Shot Detector (SSD) are fast but struggle with accuracy in complex, noisy scenes. The Detection Transformer (DETR) uses a transformer-based structure to address some of these issues. However, these models often lack the ability to aggregate and explore information between network layers, hindering their performance with complex objects. Furthermore, existing work on transparent object detection often focuses on 3D shape estimation or segmentation, requiring depth information and slower processing times unsuitable for real-time applications in automated labs. This research addresses the shortcomings of prior models by developing a novel approach.

This study introduces DenseSSD, a densely connected single-shot detector, incorporating a densely connected pyramidal layer to improve feature representation learning and object detection performance. The DenseSSD architecture comprises a mainstream network (four dense blocks and four transition layers) and a pyramidal feature cascading structure (six feature blocks and five reduction layers). The densely connected mechanism enhances information flow and feature aggregation between layers. A large-scale dataset was created using real-world images from an automated material synthesis environment. The dataset included 789 images (8764 vial cases for training and 1502 for testing), manually labeled as success (correctly placed) or failure (fall-out, lie-down, lean-in, stand-on). Data augmentation techniques (flipping, brightness/saturation/hue adjustments, Gaussian filter) were applied to the training dataset. The performance of DenseSSD was compared against benchmark models (DETR, YOLOv3, YOLOv6, SSD) using average precision (AP) and mean average precision (mAP) as evaluation metrics. The study expanded the dataset to include solution-filled vials with varying solution colors to further evaluate robustness. Additional experiments tested the model's sensitivity to camera view angles (30°, 45°, 60°, 90°), using transfer learning with pre-trained weights from the 45° dataset. Finally, the model's generalizability was evaluated across various lab settings, including different stirring machines with varying vial densities and lighting conditions. A safety alert module was integrated to immediately notify researchers of any detected failures via remote notification systems.

DenseSSD significantly outperformed the benchmark models. It achieved a mAP of 95.2% on the complex dataset with both empty and solution-filled vials, surpassing YOLOv6 (86.6%), SSD (82.1%), DETR (83.9%), and YOLOv3 (61.4%). The improvement was particularly striking for failure case detection, reaching 90.5% AP, crucial for minimizing accidents. DenseSSD showed superior computational efficiency with fewer parameters and FLOPS compared to the other models. The precision-recall (PR) curve analysis demonstrated higher stability (AUC = 0.97) in DenseSSD, minimizing false alarms. The model's robustness to environmental variations was confirmed through experiments using solution-filled vials with varying colors and different camera view angles. Even with view angle changes (30°, 45°, 60°, 90°), DenseSSD maintained high mAPs (88.5%, 94.8%, 93.8%, 84.9% respectively), showcasing its resilience to environmental factors. Experiments in unconstrained environments with various stirring machine types (sparse, semi-sparse, dense) and lighting conditions (bright, dark) also demonstrated DenseSSD's superior performance and adaptability. Feature map visualizations revealed that DenseSSD generated clearer and more detailed feature maps, facilitating accurate vial positioning detection, particularly in differentiating transparent objects from noisy backgrounds.

DenseSSD effectively addresses the research question by providing a highly accurate and robust object detection system for transparent chemical vessels in automated material synthesis laboratories. The superior performance of DenseSSD compared to existing models is attributed to the densely connected mechanism and pyramidal feature cascading structure. This allows DenseSSD to effectively extract and aggregate relevant features from multiple layers, improving its ability to differentiate between different vial positions, even in challenging conditions. The high detection accuracy for failure cases significantly enhances safety by minimizing the likelihood of accidents. The robustness to environmental changes demonstrated through various experiments suggests that DenseSSD is readily adaptable to diverse laboratory settings, promoting wider implementation in automated chemical synthesis workflows. The success of DenseSSD in handling both empty and solution-filled vials, different camera angles, and unconstrained laboratory settings highlights its practical utility and generalizability.

This research successfully developed DenseSSD, a highly accurate and robust deep learning-based object detector for transparent chemical vessels in automated material synthesis. DenseSSD's superior performance and adaptability make it a valuable tool for enhancing safety in automated laboratories. Future research could explore the integration of DenseSSD into more sophisticated robotic control systems and its application to other types of laboratory equipment and tasks.

The current dataset focuses on a standard type of 20ml vial. Further research is needed to evaluate DenseSSD's performance with different vial types and sizes. While the study included data augmentation to account for varying conditions, additional data collection in diverse laboratory environments would further strengthen the model's generalizability. The study's reliance on a hard-coded robotic arm movement path may limit the direct applicability to more flexible automation systems; however, the results suggest that DenseSSD can be adapted with appropriate transfer learning.