Our Research

The following presents my comprehensive research experience and areas of focus, along with a timeline highlighting the periods when I was most actively engaged in each field. The template is from here.

• Video Understanding

  • Long Video with Memory MovieChat
  • Long Video with Gated Memory MovieChat+
  • Video Detailed Captioning AuroraCap
  • Lecture Benchmark Video-MMLU
• Spatial Understanding

  • Efficiency via Token Merging DTC
  • Visual Grounding 3DVG-LLM
• Embodied Agent

  • Build in Minecraft STEVE-1
  • Multi-Agent STEVE-1.5
  • + Knowledge Distillation STEVE-2
  • Embodied Mobile Manipulation EMMOE

• Image

  • Style Transfer in Fassion Diffashion
  • Restoration with Diffusion Prior DTPM
  • + Reinforcement Learning VersaT2I
  • Benchmark Science-T2I
  • + Large Multi-modal Model Dream Engine
• Video

  • Video Editing with Layered Representation StableVideo
• 3D World

  • Single Object Consist3D
  • City Layout in BEV CityGen
  • City with Graphics Engine CityCraft

• 3D Human Pose Estimation

  • + Domain Adaptation PoseDA
  • + Domain Adaptation and Diffusion PoSynDA
  • + Diffusion ZeDO
  • + Domain Adaptation and Diffusion ZeDO-i
  • Animal Pose Estimation UniAP
  • Use 4D Radar Tensor RT-Pose
• Motion Representation

  • Motion Generation PackDiT
  • Mask Modeling MPM
  • Contrastive Learning UniHPE
• Tracking

  • + mamba MambaMOT
  • Ego-Centric in 3D Space Ego3DT
  • + SAM SAMURAI