Computer Science > Computer Vision and Pattern Recognition
Title:MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer
Abstract:Motion transfer enables controllable video generation by transferring temporal dynamics from a reference video to synthesize a new video conditioned on a target caption. However, existing Diffusion Transformer (DiT)-based methods either transfer the motion of a single object or handle multiple objects without binding each object caption to its object, restricting fine-grained control in real-world scenes. In this work, we introduce MotionGrounder, a training-free DiT-based framework that is the first to jointly handle motion transfer and object grounding in multi-object scenes. Our Flow-based Motion Signal (FMS) in MotionGrounder provides a stable motion prior for target video generation, while our Object-Caption Alignment Loss (OCAL) grounds object captions to their corresponding spatial regions. We further propose two new metrics: the Object Grounding Score (OGS), which jointly evaluates (i) spatial alignment between source video objects and their generated counterparts and (ii) semantic consistency between each generated object and its target caption, and the Motion Grounding Score (MGS), which evaluates whether each object moves like its source object at the correct location. Our experiments show that MotionGrounder improves object grounding over recent baselines, with the advantage appearing once several objects are present, across quantitative, qualitative, and human evaluations.
| Comments: | Accepted at NeurIPS 2026. Please visit our project page at this https URL |
|---|---|
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2604.00853 [cs.CV] |
| (or arXiv:2604.00853v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.00853 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
- HTML (experimental)
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.