Newshot

OMINI: Towards Consistent 3D Object Manipulation via Generative Texturing and Adaptive Lighting

Summary

arXiv:2501.12935v2 Announce Type: replace Abstract: Enabling object manipulation based on a single static image, from geometric editing to animation, remains a significant challenge. Existing methods often struggle to jointly support these tasks, while also suffering from fidelity degradation in object appearance and scene illumination. In this work, we introduce OMINI, a unified framework that addresses these challenges through two complementary strategies. First, OMINI lifts the 2D object into a 3D representation and adopts a Refine-before-Render design. Both geometric editing and animation are then performed by manipulating and rendering the refined 3D asset, maintaining appearance consistency throughout the manipulation process. Second, we propose two dedicated modules for fidelity enhancement: TextuRefiner refines rendered views of the 3D asset under appearance and geometric guidance, thereby generating fine-grained texture details; IllumiCorrector corrects environmental illuminat…

Original text(via arXiv cs.CV)

Computer Science > Computer Vision and Pattern Recognition

Title:OMINI: Towards Consistent 3D Object Manipulation via Generative Texturing and Adaptive Lighting

Abstract:Enabling object manipulation based on a single static image, from geometric editing to animation, remains a significant challenge. Existing methods often struggle to jointly support these tasks, while also suffering from fidelity degradation in object appearance and scene illumination. In this work, we introduce OMINI, a unified framework that addresses these challenges through two complementary strategies. First, OMINI lifts the 2D object into a 3D representation and adopts a Refine-before-Render design. Both geometric editing and animation are then performed by manipulating and rendering the refined 3D asset, maintaining appearance consistency throughout the manipulation process. Second, we propose two dedicated modules for fidelity enhancement: TextuRefiner refines rendered views of the 3D asset under appearance and geometric guidance, thereby generating fine-grained texture details; IllumiCorrector corrects environmental illumination using HSL-based color correction and VLM-guided intensity optimization, enabling visually plausible renderings with shadows. Extensive experiments demonstrate that OMINI outperforms existing methods, producing visually consistent results. The GPU stages of OMINI are run on a single NVIDIA RTX 3090.
Comments:Accepted for publication in IEEE Transactions on Image Processing (TIP). The first version is submitted to OpenReview in September 2024: this https URL
Subjects:Computer Vision and Pattern Recognition (cs.CV)
Cite as:arXiv:2501.12935 [cs.CV]
(or arXiv:2501.12935v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2501.12935 arXiv-issued DOI via DataCite

Submission history

Access Paper:

  • HTML (experimental)

Additional Features

  • Audio Summary

Current browse context:

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by arXiv cs.CV
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗