This project formulated pandemic intervention planning as an infinite-horizon Markov Decision Process over Jakarta, Samarinda, Makassar, and Sorong. In a team of three, I helped implement a stochastic Gymnasium environment with city-level SEIRD dynamics, economic and budget variables, and independent lockdown, travel-restriction, and vaccination controls.
The main environment has a 22-dimensional continuous state and approximately 5.3 million city-specific intervention combinations. Its objectives account for health, mortality, economic activity, immunity, and budget, with action clipping to keep interventions feasible. Reduced 1,024-state and 4,096-state MDPs were evaluated using Value Iteration, Policy Iteration, and Q-Learning to check the formulation on smaller problems.
PPO, A2C, and a restricted uniform-national DQN baseline were trained for 50,000 environment steps. DQN used a restricted action space rather than the full city-specific controls, so its results need to be read with that distinction in mind. PPO achieved the best mean return, 32.36 ± 12.98, across 30 held-out seeds.
In a separate set of 50 descriptive simulations, PPO eradicated the simulated outbreak in every episode while retaining an average 44.65% of the intervention budget. These are results within the course simulation, not evidence of real-world public-health effectiveness.