Artificial intelligence (AI) is increasingly important in drug discovery, with a key goal being the development of autonomous de novo drug design methods. Traditional methods combine QSAR, molecular replacements, simulations, and docking, while newer generative models leverage deep learning (variational autoencoders, GANs, reinforcement learning). Successful examples include the generation of agonists for retinoid X receptors and peroxisome proliferator-activated receptors. However, existing methods often rely on extensive experimental data, which is unavailable for novel drug targets. For instance, GENTRL, while successfully generating DDR1 inhibitors, required 46 days and used existing active molecules in its training data. Virtual screening, another approach, demands significant computational resources and expertise. This study presents MORLD, a novel deep generative model designed to overcome these limitations by integrating reinforcement learning and docking simulations, requiring only the target protein's 3D structure.

The authors review existing AI-driven drug discovery approaches, highlighting both successes and limitations. They discuss the use of deep generative models like GENTRL in de novo drug design, emphasizing its reliance on experimental data and its tendency to generate molecules similar to those in the training set. The limitations of virtual screening, particularly its computational cost and requirement for expertise, are also discussed. These limitations motivate the development of MORLD, a data-efficient and readily accessible method for de novo drug design.

MORLD (Molecule Optimization by Reinforcement Learning and Docking) combines reinforcement learning and docking simulations. The process begins with an initial molecule and a target protein structure. The core of MORLD is MolDQN, a framework that uses reinforcement learning and chemical domain knowledge to modify the molecule through a series of steps (one episode). Each step involves adding or removing an atom or bond, ensuring chemical validity using RDKit. The action selection employs a decaying epsilon-greedy method, balancing exploration and exploitation. During each episode, the molecule is evaluated using synthetic accessibility (SA) and quantitative estimate of drug-likeness (QED) scores before the final step, where QuickVina2 docking score is used. The weighted sum of SA, QED, and docking scores (at the terminal state) constitutes the reward for MolDQN. Multiple episodes are run, enabling MolDQN to learn which modifications yield higher rewards, ultimately generating molecules with improved docking scores, SA, and QED scores. A random model was developed as a control to compare against MORLD's performance.

MORLD demonstrated significant improvements in generating novel inhibitors compared to a random model. For DDR1, MORLD consistently increased docking scores, SA scores, and QED scores over the course of training episodes. While the number of unique compounds generated decreased after a certain number of episodes due to the reinforcement learning algorithm converging towards a single optimal policy, MORLD still produced compounds with docking scores comparable to or better than experimentally validated compounds from Zhavoronkov et al. Furthermore, the generated molecules showed significant diversity, as evident from their broad range of Tanimoto similarity scores to the lead compound. For D4DR, MORLD successfully generated agonists even without a starting lead molecule, showcasing its potential to replace computationally expensive virtual screening. Docking pose analysis revealed that MORLD-optimized molecules maintain key interactions with the target protein while also creating new interactions that contribute to improved docking scores. A comparison with AutoGrow4, a similar method, showed that MORLD produced a molecule with a better docking score, SA score, and QED score when optimizing the same lead compound against PARP-1. These findings highlight MORLD's efficiency and effectiveness in generating molecules with improved properties and diverse chemical space exploration.

MORLD addresses the limitations of existing de novo drug design methods by requiring minimal data (only the target protein structure) and achieving high efficiency. Its ability to generate novel inhibitors comparable to experimentally validated compounds, even starting from scratch, demonstrates its potential for accelerating drug discovery, especially for novel drug targets. The successful application to both DDR1 and D4DR, with diverse initial conditions (existing lead, virtually screened lead, and de novo design), strengthens the general applicability of MORLD. The observation that MORLD preserves and creates new favorable interactions during optimization underscores its ability to learn the principles of effective ligand binding.

MORLD offers a novel approach to autonomous molecule generation, efficiently utilizing reinforcement learning and docking simulations. It requires only the target protein structure, unlike data-intensive alternatives, and its speed and accessibility (via a public web server) makes it a valuable tool for drug discovery research. Future work could focus on addressing the limitations regarding docking simulation accuracy, the representation of 3D structural information, and the convergence of the reinforcement learning algorithm to explore a wider chemical space more effectively.

The study acknowledges several limitations. Docking scores are not a perfect predictor of binding affinity. The method's applicability is restricted by the availability of the target protein's 3D structure. The ECFP representation may not fully capture 3D structural information. The reinforcement learning algorithm tends to converge on a limited number of optimized compounds. The atom-based approach may limit the exploration of all chemically valid spaces.