arxiv:2610.29997

DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations

Published on Oct 21

· Submitted by

Ziang Cao on Oct 22

Authors:

Mingze Sun ,

Ziang Cao ,

Abstract

DONIC is a generalist bimanual tracker trained on diverse human-motion data that uses reinforcement learning with recycled demonstrations and per-hand rewards to achieve high success on unseen objects without fine-tuning.

Generated by thinkingmachines/Inkling-Small

Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.

View arXiv page View PDF Project page GitHub 25 Add to collection

Community

Caoza

Paper author Paper submitter 1 day ago

DONIC is a generalist dexterous bimanual tracking policy: one policy learns to track diverse human two-hand demonstrations with two robot hands, and generalizes to unseen trajectories and unseen objects without fine-tuning.

🔥 Highlights

🤲 Policy — One Policy, Two Hands. A single reference-conditioned policy controls two Wuji dexterous hands and tracks bimanual references across operation categories and human-motion sources.
📊 Data — BiHOI. 1,575 multi-view bimanual sequences over 229 objects and 14.77 hours of interaction, captured with a calibrated multi-view camera array and emphasizing sustained two-hand contact.
🧠 Learning — Signals Built for Two Hands. An object-centered per-hand reward, Clip-Centered Advantages, and a PPO + behavioral cloning loop that recycles replay-verified successful executions into training demonstrations.
🏆 Generalization — Zero Fine-Tuning. 86.79% success on unseen trajectories and 57.32% on unseen objects in simulation, 77.36 and 45.73 percentage points above the stronger of our bimanual adaptations of DexTrack and Dexplore.
📈 Scaling — More Human Data Helps. Adding BiHOI improves all seven metrics on both held-out splits, raising success to 88.68% and 59.76%.