NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance

Wenzhe Cai  Jiaqi Peng  Yuqiang Yang  Yujian Zhang  Meng Wei 
Hanqing Wang  Yilun Chen  Tai Wang  Jiangmiao Pang 
Shanghai AI Laboratory  Tsinghua University 
Zhejiang University  The University of Hong Kong 

🔥 News

• We release the benchmark and baselines implementations here!
• We release a high-quality VLN simulation dataset - InternNav-N1!

🏡 About

Learning navigation in dynamic open-world environments is an important yet challenging skill for robots. Most previous methods rely on precise localization and mapping or learn from expensive real-world demonstrations. In this paper, we propose a novel diffusion policy network that enables zero-shot sim-to-real transfer and can generalize across different robot platform and diverse scenes. Besides, we build a highly efficient navigation data generation pipeline that can support both synthetic scene assets and 3D Gaussian-Splatting assets.

🛠️ Installation

Please follow the instructions to config the environment for NavDP.

Step 0: Clone this repository

git clone https://github.com/wzcai99/NavDP.git
cd NavDP/navdp_api/

Step 1: Create conda environment and install the dependency

conda create -n navdp python=3.10
conda activate navdp
pip install -r requirements.txt

🤖 Run NavDP Model

Please fill this form to access the link to download the NavDP model checkpoint. Then, run the following line to start navdp server:

python navdp_server.py --port ${YOUR_PORT} --checkpoint ${SAVE_PTH_PATH}

By querying with RGB-D observations, the navdp server will return the prediction trajectories as well as the critic values. We provide 2 examlpes of RGB-D observation clips, you can download via the following link: example-A-RGB, example-A-Depth, example-B-RGB, example-B-Depth.

And we provide an example code to run the inference results with visualization. You can choose to run no-goal policy or point-goal policy for your own purpose with calling different api functions.

python navdp_client.py --port ${YOUR_PORT}  -rgb_pkl ${SAVED_RGB_PKL} --depth_pkl ${SAVED_DEPTH_PKL} --output_path ${EXPECT_OUTPUT_PATH}

📝 TODO List

• Release the arXiv paper in May, 2025.
• Release the scripts and checkpoint for deployment.
• Release the evaluation benchmark.
• Release the large-scale navigation dataset.

✉️ Contact

For any questions, please feel free to email [email protected]. We will respond to it as soon as possible.

🎉 Acknowledgments

This repository is built upon the support and contributions of the following open-source projects.

• depth_anything: The foundation representation for RGB image observations.
• diffusion_policy: The implementation of the diffusion-based robot policy.
• IsaacLab: Efficient simulation platform for building the navigation benchmark in our work.

🔗 Citation

If you find our work helpful, please cite it:

@article{cai2025navdp,
  title = {NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance},
  author = {Wenzhe Cai, Jiaqi Peng, Yuqiang Yang, Yujian Zhang, Meng Wei, Hanqing Wang, Yilun Chen, Tai Wang and Jiangmiao Pang},
  booktitle = {Arxiv},
  year = {2025},
}

📄 License

This work is under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.