THOM: Generating Physically Plausible Hand-Object Meshes From Text

Uyoung Jeong1, Yihalem Yimolal Tiruneh1, Hyung Jin Chang2, Seungryul Baek1, Kwang In Kim3
1UNIST, 2University of Birmingham, 3POSTECH
Paper arXiv Code
 

THOM is a novel photorealistic text-to-3D HOI generation pipeline that generalizes to diverse objects, hands and interactions.

Abstract

Generating photorealistic 3D hand-object interactions (HOIs) from text is important for applications like robotic grasping and AR/VR content creation. In practice, however, achieving both visual fidelity and physical plausibility remains difficult, as mesh extraction from text-generated Gaussians is inherently ill-posed and the resulting meshes are often unreliable for physics-based optimization. We present THOM, a training-free framework that generates physically plausible 3D HOI meshes directly from text prompts, without requiring template object meshes. THOM follows a two-stage pipeline: it first generates hand and object Gaussians guided by text, and then refines their interaction using physics-based optimization. To enable reliable interaction modeling, we introduce a mesh extraction method with an explicit vertex-to-Gaussian mapping, which enables topology-aware regularization. We further improve physical plausibility through contact-aware optimization and vision-language model (VLM)-guided translation refinement. Extensive experiments show that THOM produces high-quality HOIs with strong text alignment, visual realism, and interaction plausibility.

Framework

Framework Figure

THOM framework adopts a two-stage pipeline for generating realistic hand-object interactions. Initially, object and hand meshes are independently generated with high visual realism. In the second stage, we jointly optimize their interaction parameters using physics-based regularization losses to ensure plausible contacts and minimal penetration.

 

Comparison with Text-To-3D Methods

Comparison Figure

Qualitative comparison of our method with Text-to-3D generation methods.

 

Comparison with Text-To-HOI Methods

HOI Comparison Figure

Qualitative comparison of our method with Text-to-HOI generation methods.

 

More Qualitative Results

More Qualitative Results Figure

Additional qualitative results generated by our method.

BibTeX

@InProceedings{Jeong_2026_CVPR,
    author    = {Jeong, Uyoung and Tiruneh, Yihalem Yimolal and Chang, Hyung Jin and Baek, Seungryul and Kim, Kwang In},
    title     = {THOM: Generating Physically Plausible Hand-Object Meshes From Text},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
    month     = {June},
    year      = {2026},
    pages     = {3653-3664}
}