Bench2Dex
A Bimanual Manipulation Benchmark across Dexterous Hands
with Simulated Fingertip Visuo-Tactile Support
1Shanghai Jiao Tong University · 2Fudan University · 3The University of Hong Kong · 4Inspire Robots
5Zhongguancun Academy · 6COWARobot Co. Ltd · 7Nanyang Technological University
* Core contribution · † Corresponding authors
Abstract
A bimanual manipulation benchmark with simulated visuo-tactile support.
Bimanual dexterous manipulation requires coordinated control of two hands in multi-stage, contact-rich tasks. We introduce Bench2Dex, a simulation benchmark for bimanual manipulation supporting 12 dexterous hands. The benchmark includes 26 tasks involving tool use, interaction with articulated objects, and multi-stage coordination, together with approximately 1.3K human-teleoperated demonstrations. It integrates demonstration collection, synchronized multimodal data, and automated metrics for task completion and intermediate progress within a common framework. We equip every supported hand model with simulated vision-based tactile sensing at the fingertips, encoding local contact geometry as image-like tactile observations. Through a unified visuo-tactile interface, we make these observations available alongside external vision and proprioception to support research on tactile representations and multimodal policy learning. We establish reference results using ACT, Diffusion Policy, π₀.₅, and GR00T N1.5 with visual and proprioceptive inputs. We assess robustness across four evaluation channels—None, Equivariance-only, Invariance-only, and Full—covering an unperturbed baseline and controlled combinations of seven scene perturbation types. By combining a bimanual manipulation benchmark across dexterous hands with simulated visuo-tactile support and automated policy evaluation, Bench2Dex provides the community with a common starting point for investigating when tactile information is useful and how manipulation policies can effectively incorporate it.
System Overview
One pipeline: teleoperation → multimodal acquisition → shared visuo-tactile representation → evaluation.
Comparison
Comparison with representative manipulation benchmarks.
Most manipulation benchmarks and datasets use a limited set of robot platforms, one gripper or hand design, or a narrow set of sensing modalities. Even large datasets spanning multiple embodiments lack evaluation protocols designed to compare dexterous manipulation policies across different arm–hand designs and contact surfaces. Bench2Dex connects demonstration collection, policy learning, and evaluation within a common framework.
| Benchmark | Task Num | Embodiment | Tool/Device Use | Dexterous Hand | Teleoperation | Vision-Based Tactile | Articulated |
|---|---|---|---|---|---|---|---|
| LIBERO | 130 | 1 | ✗ | ✗ | ✓ | ✗ | ✓ |
| RLBench2 | 13 | 1 | ✓ | ✗ | ✗ | ✗ | ✓ |
| RoboCasa | 100 | 1 | ✓ | ✗ | ✓ | ✗ | ✓ |
| DROID | 86 | 1 | ✗ | ✗ | ✓ | ✗ | ✗ |
| DexMimicGen | 9 | 3 | ✗ | ✓ | ✓ | ✗ | ✓ |
| RealMirror | 5 | 1 | ✓ | ✓ | ✓ | ✗ | ✓ |
| RoboTwin 2.0 | 50 | 5 | ✓ | ✗ | ✗ | ✗ | ✓ |
| RoboMIND 2.0 | 739 | 6 | ✓ | ✓ | ✓ | ✗ | ✓ |
| MuJoCo Manipulus | 16 | 1 | ✓ | ✗ | ✗ | ✗ | ✗ |
| RoboCasa365 | 365 | 1 | ✓ | ✗ | ✓ | ✗ | ✓ |
| RoboDojo | 60 | 3 | ✓ | ✗ | ✓ | ✗ | ✓ |
| BiCoord | 18 | 1 | ✗ | ✗ | ✗ | ✗ | ✗ |
| DexJoCo | 11 | 2 | ✓ | ✓ | ✓ | ✗ | ✓ |
| DexVerse∗ | 19 | 1 | ✓ | ✓ | ✓ | ✗ | ✓ |
| Bench2Dex (Ours) | 26 | 12 | ✓ | ✓ | ✓ | ✓ | ✓ |
Note. ∗ Statistics based on publicly available data.
Embodiment Atlas · 12 Arm–Hand Morphologies
One benchmark, twelve arm–hand morphologies.
Bench2Dex supports 12 dexterous hand models with a common teleoperation framework and a unified visuo-tactile interface. The interface provides image-like observations of local contact geometry while preserving hand-specific sensing geometry across different arm–hand designs.
Teleoperation Pipeline
From human hands to simulated bimanual dexterity.
Manus gloves and ARKit wrist tracking control all 12 embodiments through a shared retargeting interface.
Capture Wrist and Hand Motion
A Manus glove provides a 25-node hand skeleton via shared memory; an ARKit stream provides wrist translation and orientation.
Retarget Skeleton and Solve Arm IK
The skeleton is converted to 21 MediaPipe-style keypoints; DexPilot retargeting solves hand joints, a Pinocchio IK controller solves the arm.
Record Actions, States, Observations, Traces
Per step, the commanded action is stored, then post-step observations and evaluator state are written to a time-aligned HDF5 episode.
Eight Synchronized Data Modalities
Eight time-aligned views of interaction.
Explore eight synchronized sensor and state modalities from one recorded episode. Play the video or drag the timeline to inspect individual frames.
Unified Visuo-Tactile Interface
Different hands, one unified visuo-tactile interface.
Bench2Dex equips every supported hand model with simulated vision-based tactile sensing at the fingertips. The unified visuo-tactile interface provides image-like observations of local contact geometry alongside external vision and proprioception, while retaining hand-specific sensing geometry. This supports research on when tactile information is useful and how policies can effectively incorporate it.
An 8-bit tactile image Ts,t ∈ {0,…,255} is produced per tactile site per frame: ray-cast contact distance is piecewise-quantized (0.005 mm/level below 0.5 mm, 0.03 mm/level above, saturating near 5.15 mm → 255) for fine near-contact sensitivity, then Gaussian-smoothed.
Task Suite · 26 Long-Horizon Tasks
The long-horizon task suite.
Each task has multiple stages and a machine-checkable success condition.
Executable Evaluation
From “success” to an executable protocol.
A task is successful only when its executable terminal predicate remains satisfied for the required dwell time. The evaluation suite measures task completion, stage-level progress, and success-conditioned efficiency under a fixed protocol, step budget, dwell time, random seed, and sampled generalization parameters.
Example: Microwave Bowl Loading is evaluated through four executable stages; the dwell latch is what makes “closed” a success rather than a transient.
Controlled Generalization
Four channels, seven controlled factors.
Each evaluation keeps the semantic goal, object set, and success conditions unchanged while controlling two groups of scene factors. Invariance factors alter task-irrelevant visual conditions without changing the intended task behavior; equivariance factors alter task-relevant geometry and require corresponding changes in reaching, grasping, and contact trajectories.
None / Equi. / Inv. use matched anchors aligned by episode index; Full is sampled independently without an anchor.
Results
Policy Leaderboard
Reported policy evaluations across 26 tasks.
Get Started
From a scene to an evaluation.
Environment setup, policy training and evaluation, and the full 26-task catalog are in the Bench2Dex Documentation.