My name is Zhancun Mu, or 牟湛存 in Chinese. You can contact me at muzhancun@stu.pku.edu.cn. I am currently a senior student in Tong Class, Peking University. I am an incoming Ph.D. candidate at the Institute for Artificial Intelligence, Peking University starting from 2025, under the supervision of Professor Yitao Liang.

My research is focused on developing autonomous agents capable of operating in open-ended, dynamic environments such as Minecraft. To achieve this goal, I concentrate on two key elements: First, designing robust planning systems that enable agents to make complex decisions over long time horizons. This involves developing algorithms for reasoning about uncertain outcomes, decomposing high-level goals into executable subgoals, and continuously revising plans as new information becomes available. Second, creating intuitive controllers that allow agents to seamlessly interact with their environments. This requires integrating perception, decision-making, and low-level control in a unified framework that can handle the rich, multimodal inputs and outputs characteristic of virtual worlds. In addition to these core areas, I am keenly interested in multi-agent systems, exploring how autonomous agents can coordinate their behaviors to accomplish shared objectives. I am also drawn to cognitive reasoning and developing computational models that capture aspects of human-like intelligence. Furthermore, I am excited by the potential of AI to accelerate scientific discovery across domains. My ultimate vision is to develop intelligent agents that can autonomously explore, understand, and shape the open-ended environments they inhabit in pursuit of complex, self-motivated goals.

🔥 News

• Oct 2024: 🎉🎉 Our paper ‘‘ROCKET-1: Mastering Open-Wolrd Interaction with Visual-Temporal Context Prompting’’ has been realeased!
• Sep 2024: 🎉🎉 Our latest Vision-Language-Action model OmniJARVIS is accepted by NeurIPS 2024.
• Jun 2024: 🎉🎉 Our paper “A Contextual Combinatorial Bandits Approach to Negotiation” is accepted at ICML 2024.
• Jan 2024: 🎉🎉 PTGM is accepted by ICLR 2024 for oral presentation (top 1.2%).

📚 Preprints

arXiv

MineStudio: A Streamlined Package for Minecraft AI Agent Development

Shaofei Cai*, Zhancun Mu*, Kaichen He, Bowei Zhang, Xinyue Zheng, Anji Liu, Yitao Liang

arXiv | Paper | Doc | Code | PyPI | Dataset

arXiv

GlobalTomo: A global dataset for physics-ML seismic wavefield modeling and full-waveform inversion

Shiqian Li, Zhi Li, Zhancun Mu, Shiji Xin, Zhixiang Dai, Kuangdai Leng, Ruihua Zhang, Xiaodong Song, Yixin Zhu

arXiv | Paper | Page | Code

📝 Publications

NeurIPS 2024-OWA

ROCKET-1: Master Open-World Interaction with Visual-Temporal Context Prompting

Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

NeurIPS 2024 Open-World Agent Workshop | Paper | Cite | Code | Page

NeurIPS 2024

OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents

Zihao Wang, Shaofei Cai, Zhancun Mu, Haowei Lin, Ceyao Zhang, Xueije Liu, Qing Li, Anji Liu, Xiaojian Ma, Yitao Liang

NeurIPS 2024 | Paper | Twitter | Page

ICML 2024

A Contextual Combinatorial Bandit Approach to Negotiation

Yexin Li, Zhancun Mu, Siyuan Qi

ICML 2024 | Paper

ICLR 2024

Pre-Training Goal-based Models for Sample-Efficient Reinforcement Learning

Haoqi Yuan, Zhancun Mu, Feiyang Xie, Zongqing Lu

Oral Presentation

ICLR 2024 | Paper | Page | Code

📖 Educations

2021.09 - now, Undergraduate

Tong class, Yuanpei College

Peking University, Beijing

💻 Internships

• 2023.07 - 2023.10, BAAI, China.