Newshot

MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer

Summary

arXiv:2604.00853v2 Announce Type: replace Abstract: Motion transfer enables controllable video generation by transferring temporal dynamics from a reference video to synthesize a new video conditioned on a target caption. However, existing Diffusion Transformer (DiT)-based methods either transfer the motion of a single object or handle multiple objects without binding each object caption to its object, restricting fine-grained control in real-world scenes. In this work, we introduce MotionGrounder, a training-free DiT-based framework that is the first to jointly handle motion transfer and object grounding in multi-object scenes. Our Flow-based Motion Signal (FMS) in MotionGrounder provides a stable motion prior for target video generation, while our Object-Caption Alignment Loss (OCAL) grounds object captions to their corresponding spatial regions. We further propose two new metrics: the Object Grounding Score (OGS), which jointly evaluates (i) spatial alignment between source video ob…

Original text(via arXiv cs.CV)

Computer Science > Computer Vision and Pattern Recognition

Title:MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer

Abstract:Motion transfer enables controllable video generation by transferring temporal dynamics from a reference video to synthesize a new video conditioned on a target caption. However, existing Diffusion Transformer (DiT)-based methods either transfer the motion of a single object or handle multiple objects without binding each object caption to its object, restricting fine-grained control in real-world scenes. In this work, we introduce MotionGrounder, a training-free DiT-based framework that is the first to jointly handle motion transfer and object grounding in multi-object scenes. Our Flow-based Motion Signal (FMS) in MotionGrounder provides a stable motion prior for target video generation, while our Object-Caption Alignment Loss (OCAL) grounds object captions to their corresponding spatial regions. We further propose two new metrics: the Object Grounding Score (OGS), which jointly evaluates (i) spatial alignment between source video objects and their generated counterparts and (ii) semantic consistency between each generated object and its target caption, and the Motion Grounding Score (MGS), which evaluates whether each object moves like its source object at the correct location. Our experiments show that MotionGrounder improves object grounding over recent baselines, with the advantage appearing once several objects are present, across quantitative, qualitative, and human evaluations.
Comments:Accepted at NeurIPS 2026. Please visit our project page at this https URL
Subjects:Computer Vision and Pattern Recognition (cs.CV)
Cite as:arXiv:2604.00853 [cs.CV]
(or arXiv:2604.00853v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2604.00853 arXiv-issued DOI via DataCite

Submission history

Access Paper:

  • HTML (experimental)

Additional Features

  • Audio Summary

Current browse context:

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by arXiv cs.CV
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗