The exponential growth in publications containing chemical structures contrasts sharply with the limited availability of this information in machine-readable formats within public databases. Manual extraction is time-consuming, error-prone, and insufficient for deep learning applications. Optical Chemical Structure Recognition (OCSR) aims to automate this process. While several OCSR tools exist, most are proprietary or unavailable, hindering widespread use and progress in fields like medicine, pharmacy, materials science, and molecular biology which heavily rely on chemical data. DECIMER.ai addresses this limitation by providing a comprehensive, open-source workflow for identifying, segmenting, and recognizing chemical structure depictions in scientific literature, leveraging recent advances in deep learning, computer vision, and natural language processing. The lack of openly available tools capable of combining segmentation, classification, and translation within a unified workflow motivates the development of DECIMER.ai.

Traditional OCSR tools often rely on rule-based approaches and struggle with distorted images. Deep learning-based methods have emerged as more robust alternatives, yet many remain proprietary or inaccessible. Existing open-source solutions often lack a complete workflow encompassing segmentation, classification, and structure translation. This paper reviews existing OCSR solutions, highlighting the limitations of proprietary and incomplete open-source alternatives, setting the stage for the introduction and justification of DECIMER.ai as a superior and openly accessible solution.

DECIMER.ai employs a three-stage pipeline: 1) **DECIMER Segmentation**, based on Mask R-CNN, identifies and segments chemical structures within images. 2) **DECIMER Image Classifier**, using a fine-tuned EfficientNet-V1-B0 model, distinguishes images containing chemical structures from other images. 3) **DECIMER Image Transformer**, the core component, utilizes an EfficientNet-V2-M CNN encoder and a Transformer decoder to translate segmented images into SMILES strings. The model was trained on a massive dataset (>450 million depictions) generated using RanDepict, a tool capable of generating diverse chemical structure depictions with a range of depiction styles and augmentations from various cheminformatics toolkits (CDK, RDKit, Indigo, and PIKACHU). The training process involved extensive image augmentation to improve model robustness. Data preprocessing included filtering molecules (molecular weight < 1500 Da), removal of explicit hydrogens, and handling of Markush structures. SMILES strings were tokenized using a Keras tokenizer. The model was trained using the Adam optimizer with a custom learning rate scheduler on Google Cloud Platform TPUs. The web application itself was developed using Laravel 8 and deployed as a three-container Docker application to facilitate parallel processing of images for faster user experience. Model performance was evaluated using in-domain and benchmark datasets, assessing metrics such as identical prediction proportion, average Tanimoto similarity, and BLEU scores. The benchmark included comparing DECIMER against other publicly available OCSR tools (OSRA, MolVec, Imago, Img2Mol, SwinOCSR, and MolScribe) on standard datasets (JPO, CLEF, USPTO, UOB, Indigo, USPTO Big, and Img2Mol test set), both with and without added distortions. The impact of different R-group representations in SMILES was also analyzed and addressed by modifying the SMILES to avoid ambiguity.

DECIMER Image Transformer significantly outperforms other open-source OCSR tools on various benchmark datasets, achieving high accuracy and low failure rates, even with distorted images. DECIMER Image Classifier demonstrates high accuracy in distinguishing chemical structure depictions from other images across different datasets, including real-world images from a chemical conference. The average Tanimoto similarity of DECIMER Image Transformer predictions was over 0.95, indicating high similarity to the original molecules. The system also demonstrates robust performance on Markush structures and hand-drawn images, even though these were not explicitly present in the training data. The study showed that using modified SMILES representations for Markush structures, where ambiguous tokens are replaced, improved model accuracy. DECIMER Image Transformer V2 exhibited substantially improved performance compared to its predecessor (V1), highlighting the effectiveness of advancements in data generation and model optimization. Detailed benchmark results, including the proportion of identical predictions (Pi), average Tanimoto similarity (T), catastrophic and severe failure rates, are presented in tables, comparing DECIMER's performance against OSRA, MolVec, Imago, Img2Mol, SwinOCSR, and MolScribe across various datasets and conditions. The impact of different image resolutions (299x299 vs. 512x512 pixels) on the recognition of Markush structures was also assessed.

DECIMER.ai's superior performance across benchmark datasets and its ability to handle diverse image types (including Markush structures and hand-drawn images) demonstrate its robustness and generalizability. The open-source nature of the platform enables collaborative development and improvement, potentially leading to further advancements in chemical information extraction. The platform's ability to process real-world images captured using a smartphone highlights its practicality and usability. A comparison with MolScribe reveals that while MolScribe shows excellent performance despite a significantly smaller training dataset, its use of predefined rules contrasts with DECIMER's purely data-driven approach; the latter's superior performance highlights the importance of massive datasets for data-driven approaches. The high accuracy and low failure rates achieved by DECIMER are crucial for automating the tedious and error-prone process of extracting chemical information from scientific publications, thereby accelerating research progress and enabling data-driven discoveries across various scientific disciplines. The success underscores the power of large, diverse training datasets combined with effective model architectures in achieving robust OCSR.

DECIMER.ai represents a significant advancement in OCSR, providing a comprehensive, open-source, and high-performing platform for extracting chemical structure information from scientific literature. Its superior performance, accessibility, and extensibility positions it as a valuable tool for researchers and database curators. Future work could focus on integrating text-mining capabilities to link structural information with other chemical data extracted from publications and exploring the use of even larger datasets and advanced model architectures for further performance enhancement.

While DECIMER.ai demonstrates excellent performance, potential limitations include its reliance on the quality of the input images. Extremely blurry or distorted images may still pose challenges for accurate recognition. The current system prioritizes speed and user experience, resulting in a restriction on the number of pages and structures processed through the online application. Further research into handling extremely complex or unusual chemical structures remains a potential avenue for improvement.