Pandemics pose a significant global challenge, demanding strategies that balance public health with economic stability. While vaccine development is crucial, controlling the spread of disease through interventions like lockdowns is necessary in the interim. However, these interventions often have severe economic consequences. Existing lockdown strategies, such as age-based lockdowns and repeated n-work-m-lockdown cycles, may not be optimal in all situations. This research addresses the need for improved strategies by applying reinforcement learning (RL) to a simulated pandemic environment. RL, a branch of artificial intelligence, allows an agent (trained via Deep Reinforcement Learning or DRL) to learn optimal policies for interacting with an environment to maximize rewards. The research aims to answer critical questions: Is prolonged lockdown the only effective mitigation strategy? Should lockdowns continue even when the situation is improving? How can the resurgence of a pandemic be best handled? How can economic factors be balanced while mitigating a pandemic? This study uses DRL within a virtual pandemic environment to explore these questions and discover potentially more effective control strategies.

Traditional epidemiological modeling often utilizes compartmental models such as SEIR (Susceptible-Exposed-Infectious-Recovered) models based on ordinary differential equations (ODEs). While these models are useful for understanding disease dynamics, they often lack the randomness and dynamism necessary for training RL agents. The authors note that RL has demonstrated success in various domains, including game playing and chatbot development, suggesting its potential for addressing complex real-world challenges such as pandemic control. Previous RL methods often relied on tabular functions, but advances in deep learning have enabled the use of deep neural networks (DNNs) for improved performance in DRL.

The researchers developed a virtual environment simulating pandemic spread based on an SEIR model. This environment incorporates randomness in infection, recovery, and death to prevent overfitting. The environment features a 2D grid where a population of susceptible and infectious individuals move randomly. The agent's actions consist of three levels of movement restrictions: level 0 (no restrictions), level 1 (social distancing), and level 2 (lockdown). The environment provides the agent with a state vector containing seven parameters: active cases, newly infected, cured cases, death cases, reproduction rate, economic state, and current movement restrictions. The agent's goal is to learn an optimal policy that minimizes deaths, maintains economic stability, and reduces active cases. A reward function is designed to incentivize the agent to achieve this balance. The reward function considers the current economy, cumulative deaths, and active cases, penalizing high death rates and low economic activity. A memory-based DDQN (Double Deep Q-Network) agent with three bidirectional LSTM layers followed by four dense layers is used. Bidirectional LSTMs are employed to utilize both past and future information. The agent is trained over 7000 episodes with decaying exploration rate, a discount factor of 0.9, and MSE loss.

The virtual environment's performance was evaluated and compared with an ODE-based SEIR model, showing reasonable agreement. The reproduction rate in the virtual environment showed a close match to estimates for COVID-19. Experiments were conducted with agents having different memory lengths (7, 15, 30, 45, and 60 days). The agent with a 30-day memory (M30) showed optimal performance in terms of minimizing deaths and maximizing economic benefits. The analysis of the agent's actions revealed that the M30 agent often used a combination of prolonged initial lockdowns followed by shorter, cyclical lockdowns to control surges in active cases. The agent's decisions were influenced by both active case percentages and reproduction rates, with lockdowns generally implemented when both values were high. The agent's policy proved superior to a traditional n-work-m-lockdown policy, demonstrating its effectiveness in controlling both the initial surge and resurgences of the pandemic. Simulations showed the negative economic impacts of various lockdown approaches, highlighting the importance of strategic interventions. Comparing the agent’s actions with other models (M7, M15, M45, and M60), the agent M30 model resulted in less infections and deaths while achieving better economic stability.

The findings demonstrate the potential of DRL for developing improved pandemic control strategies. The agent's ability to balance economic considerations with the need to reduce infections provides valuable insights. The agent's sophisticated control sequences, utilizing both prolonged and cyclical lockdowns, offer a more nuanced approach than simplistic strategies. The superior performance of the agent-based policy compared to the traditional n-work-m-lockdown policy highlights the potential for AI-driven decision-making in pandemic management. This study bridges the gap between epidemiological modeling and AI, suggesting a new paradigm for pandemic preparedness and response.

This research successfully demonstrates the application of reinforcement learning to optimize pandemic control strategies. The developed virtual environment accurately simulates pandemic dynamics, and the trained agent effectively balances health and economic outcomes. The agent's strategy of employing a combination of prolonged and cyclical lockdowns proves more effective than traditional methods, showcasing AI's potential for pandemic management. Future work could explore incorporating more complex factors, such as vaccination rates, population heterogeneity, and regional variations in transmission dynamics, to further refine the model and generate even more effective control strategies.

The study's limitations include the use of a simplified model of pandemic spread and economic factors. The virtual environment, while sophisticated, doesn't capture the full complexity of real-world situations, such as variations in individual behavior, healthcare capacity, and political factors. The reward function may need further refinement to better reflect the complexities of societal trade-offs. Extending the model to incorporate real-world data and more detailed economic models would be beneficial for future research.