Computer Science

Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search

N. Dainese, M. Alakuijala, et al.

This work presents Code World Models—world models generated as Python code by LLMs for model-based RL—alongside GIF-MCTS, a new code-generation strategy, and the Code World Models Benchmark (CWMB). GIF-MCTS outperforms baselines and yields models that enable planning with much better sample efficiency and faster inference. Research conducted by Nicola Dainese, Minttu Alakuijala, Matteo Merler, and Pekka Marttinen.