Hello! Here is Shangzhe Li, an undergraduate student from South China University of Technology. I’m currently a research intern in University of California at San Diego. I’m fortunate to have the opportunity to work with Prof. Hao Su. I’m an aviation enthusiast, a Physics & Mathematics lover and also a student majoring in Artificial Intelligence. My hometown is Guangzhou. I usually live stream on Bilibili and post articles on Zhihu. And by the way, I’m an anime lover.

My research interest includes reinforcement learning, robot learning and world models.

Links to my social media:

🔥 News

2024.03, Summer intern offer received from Su Lab, UCSD! See you in San Diego in summer if everything goes smoothly!
2023.09, 🎉🎉 Homepage has been set up.

📝 Publications

Reward-free World Models for Online Imitation Learning [Preprint]

Author: Shangzhe Li, Zhiao Huang, Hao Su

Main Contribution: We propose an online imitation learning approach that utilizes reward-free world models to address tasks in complex environments. By incorporating latent planning and dynamics learning, our model can have a deeper understanding of intricate environment dynamics. We demonstrate stable, expert-level performance on challenging tasks, including dexterous hand manipulation and high-dimensional locomotion control.

The Thirteenth International Conference on Learning Representations (ICLR 2025), under review.
Augmenting Offline Reinforcement Learning with State-only Interactions [Preprint]

Author: Shangzhe Li, Xinhua Zhang

Main Contribution: We proposed a novel data augmentation method DITS for offline RL, where state-only interactions are available with the environment. The generator based on conditional diffusion models allows high-return trajectories to be sampled, and the stitching algorithm blends them with the original ones. The resulting augmented dataset is shown to significantly boost the performance of base RL methods.

The Thirteenth International Conference on Learning Representations (ICLR 2025), under review.
Data-efficient Offline Domain Adaptation for Model-free Agents using Model-based Trajectory Stitching

Author: Shangzhe Li, Hongpeng Cao, Marco Caccamo

Main Contribution: This work improves the sampling efficiency for policy adaptation in the deployment environment by stitching the offline experiences with newly collected few-shot experiences from the new environment. The proposed stitching algorithm incorporates the dynamics information of the true-MDP with the new dataset, meanwhile increasing the data diversity and de-correlating the newly collected data. The experiments on two cases show that the pre-trained policies are improved more efficiently with higher accumulated reward by using the stitched dataset than direct fine-tuning using raw data.

IEEE International Conference on Robotics and Automation (ICRA 2025), under review.

🎖 Honors and Awards

2022 First Prize, Asia and Pacific Mathematical Contest in Modeling(APMCM)
2022 Second Prize, National Contemporary Undergraduate Mathematical Contest in Modeling(CUMCM)
2022 Successful Participant, Mathematical Contest in Modeling(MCM)
2023 Successful Participant, Mathematical Contest in Modeling(MCM)
2021 Second Prize, Baidu “Paddle Paddle” Cup
2022 First Prize, Taihu Academic Innovation Scholarship (CNY 8000)
2022 Second Prize, Taihu Science Innovation Scholarship (CNY 5000)

📖 Educations

2023.10 - 2024.07, Exchange student, Technical University of Munich.
2021.09 - now, Undergraduate student, South China University of Technology.
2018.09 - 2021.06, High school student (Physics Olympiad), Affiliated High School of South China Normal University.

Current GPA: 3.87/4.00 Current Rank: 3/80

💬 Talks

2023.09, Invited by the Artificial Intelligence Association of South China University of Technology, Application of Diffusion Model on Offline Reinforcement Learning. Link: Application of Diffusion Model on Offline Reinforcement Learning
2023.12, Performed presentation in the Doctoral Seminar of Thuerey’s Group from Technical University of Munich, Application of Diffusion Model on Offline Reinforcement Learning.

💻 Internships and Research Experience

2022.09 - 2023.04, Undergraduate Research, Neural Networks Compression and Acceleration Research. Supervisor: Prof. Ye Liu, South China University of Technology.
2023.05 - 2024.01, Research Intern (Remote), Application of Diffusion Model on Offline Reinforcement Learning. Supervisor: Prof. Xinhua Zhang, University of Illinois Chicago.
2023.10 - 2024.06, Research Intern, Research on the Control Approach for Two-way Coupled Fluid Simulation. Supervisor: Prof. Nils Thuerey, Technical University of Munich, Mentor: Patrick Schnell.
2024.01 - 2024.09, Research Intern, Research on the Fast Adaptation Methods on Reinforcement Learning. Supervisor: Prof. Marco Caccamo, Technical University of Munich, Collaborator: Hongpeng Cao.
2024.02 - now, Research Intern (Remote), Distillation in LLMs via Reinforcement Learning Algorithms. Supervisor: Prof. Xinhua Zhang, University of Illinois Chicago, Collaborator: Zishun Yu.
2024.06 - now, Research Intern, Research in World Models and Inverse Reinforcement Learning. Supervisor: Prof. Hao Su, University of California, San Diego, Mentor: Zhiao Huang.

📝 Blog Articles

Notice: All of the articles here are written in Chinese.

Physics Part:

Mathematics Part:

Convex Optimization Part:

Shangzhe Li (李尚哲)