Bench2Dex
A Bimanual Manipulation Benchmark across Dexterous Hands
with Simulated Fingertip Visuo-Tactile Support
1Shanghai Jiao Tong University · 2Fudan University · 3The University of Hong Kong · 4Inspire Robots
5Zhongguancun Academy · 6COWARobot Co. Ltd · 7Nanyang Technological University
* Core contribution · † Corresponding authors
Abstract
A bimanual manipulation benchmark with simulated visuo-tactile support.
Bimanual dexterous manipulation requires coordinated control of two hands in multi-stage, contact-rich tasks. We introduce Bench2Dex, a simulation benchmark for bimanual manipulation supporting 12 dexterous hands. The benchmark includes 26 tasks involving tool use, interaction with articulated objects, and multi-stage coordination, together with approximately 1.3K human-teleoperated demonstrations. It integrates demonstration collection, synchronized multimodal data, and automated metrics for task completion and intermediate progress within a common framework. Taking a deliberate step beyond vision-only sensing, we equip every supported hand model with simulated vision-based tactile sensing at the fingertips, encoding local contact geometry as image-like tactile observations. Through a unified visuo-tactile interface, we make these observations available alongside external vision and proprioception to support research on tactile representations and multimodal policy learning. We establish reference results using ACT, Diffusion Policy, π₀.₅, and GR00T N1.5 with visual and proprioceptive inputs. We assess robustness across four evaluation channels—None, Equivariance-only, Invariance-only, and Full—covering an unperturbed baseline and controlled combinations of seven scene perturbation types. By combining a bimanual manipulation benchmark across dexterous hands with simulated visuo-tactile support and automated policy evaluation, Bench2Dex provides the community with a common starting point for investigating when tactile information is useful and how manipulation policies can effectively incorporate it.
System Overview
One pipeline: teleoperation → multimodal acquisition → shared visuo-tactile representation → evaluation.
Comparison
Comparison with representative manipulation benchmarks.
Most manipulation benchmarks and datasets use a limited set of robot platforms, one gripper or hand design, or a narrow set of sensing modalities. Even large datasets spanning multiple embodiments lack evaluation protocols designed to compare dexterous manipulation policies across different arm–hand designs and contact surfaces. Bench2Dex connects demonstration collection, policy learning, and evaluation within a common framework.
| Benchmark | Task Num | Embodiment | Tool/Device Use | Dexterous Hand | Teleoperation | Vision-Based Tactile | Articulated |
|---|---|---|---|---|---|---|---|
| LIBERO | 130 | 1 | ✗ | ✗ | ✓ | ✗ | ✓ |
| RLBench2 | 13 | 1 | ✓ | ✗ | ✗ | ✗ | ✓ |
| RoboCasa | 100 | 1 | ✓ | ✗ | ✓ | ✗ | ✓ |
| DROID | 86 | 1 | ✗ | ✗ | ✓ | ✗ | ✗ |
| DexMimicGen | 9 | 3 | ✗ | ✓ | ✓ | ✗ | ✓ |
| RealMirror | 5 | 1 | ✓ | ✓ | ✓ | ✗ | ✓ |
| RoboTwin 2.0 | 50 | 5 | ✓ | ✗ | ✗ | ✗ | ✓ |
| RoboMIND 2.0 | 739 | 6 | ✓ | ✓ | ✓ | ✗ | ✓ |
| MuJoCo Manipulus | 16 | 1 | ✓ | ✗ | ✗ | ✗ | ✗ |
| RoboCasa365 | 365 | 1 | ✓ | ✗ | ✓ | ✗ | ✓ |
| RoboDojo | 60 | 3 | ✓ | ✗ | ✓ | ✗ | ✓ |
| BiCoord | 18 | 1 | ✗ | ✗ | ✗ | ✗ | ✗ |
| DexJoCo | 11 | 2 | ✓ | ✓ | ✓ | ✗ | ✓ |
| DexVerse∗ | 19 | 1 | ✓ | ✓ | ✓ | ✗ | ✓ |
| Bench2Dex (Ours) | 26 | 12 | ✓ | ✓ | ✓ | ✓ | ✓ |
Note. ∗ Statistics based on publicly available data.
In this comparison, Bench2Dex jointly supports diverse bimanual dexterous embodiments, teleoperated demonstration collection, vision-based tactile sensing, tool/device use, and articulated-object interaction. The matrix reproduces the paper’s benchmark-comparison table.
Embodiment Atlas · 12 Arm–Hand Morphologies
One benchmark, twelve arm–hand morphologies.
Bench2Dex supports 12 dexterous hand models with a common teleoperation framework and a unified visuo-tactile interface. The interface provides image-like observations of local contact geometry while preserving hand-specific sensing geometry across different arm–hand designs.
Teleoperation Pipeline
From human hands to simulated bimanual dexterity.
A Manus glove and ARKit wrist stream drive every embodiment through the same retargeting interface, so the 12 hands are controlled by one protocol. The commanded action is recorded before the next simulation step; the resulting post-step observations and evaluator state are retained in a time-aligned unified HDF5 episode.
Capture Wrist and Hand Motion
A Manus glove provides a 25-node hand skeleton via shared memory; an ARKit stream provides wrist translation and orientation.
Retarget Skeleton and Solve Arm IK
The skeleton is converted to 21 MediaPipe-style keypoints; DexPilot retargeting solves hand joints, a Pinocchio IK controller solves the arm.
Record Actions, States, Observations, Traces
Per step, the commanded action is stored, then post-step observations and evaluator state are written to a time-aligned HDF5 episode.
Eight Synchronized Data Modalities
Eight time-aligned views of interaction.
One episode, time-aligned visual, geometric, proprioceptive, and visuo-tactile observations. The commanded action precedes the next simulation step, and the resulting post-step observations are retained with evaluator state in one HDF5 episode. Press play to watch all eight views in sync; drag the timeline to scrub.
Unified Visuo-Tactile Interface
Different hands, one unified visuo-tactile interface.
Bench2Dex equips every supported hand model with simulated vision-based tactile sensing at the fingertips. The unified visuo-tactile interface provides image-like observations of local contact geometry alongside external vision and proprioception, while retaining hand-specific sensing geometry. This supports research on when tactile information is useful and how policies can effectively incorporate it.
An 8-bit tactile image Ts,t ∈ {0,…,255} is produced per tactile site per frame: ray-cast contact distance is piecewise-quantized (0.005 mm/level below 0.5 mm, 0.03 mm/level above, saturating near 5.15 mm → 255) for fine near-contact sensitivity, then Gaussian-smoothed.
Task Suite · 26 Long-Horizon Tasks
The long-horizon task suite.
Each task is multi-stage, executable, and anchored to an executable terminal predicate.
Executable Evaluation
From “success” to an executable protocol.
A task is successful only when its executable terminal predicate remains satisfied for the required dwell time. The evaluation suite measures task completion, stage-level progress, and success-conditioned efficiency under a fixed protocol, step budget, dwell time, random seed, and sampled generalization parameters.
Example: Microwave Bowl Loading is evaluated through four executable stages; the dwell latch is what makes “closed” a success rather than a transient.
Controlled Generalization
Four channels, seven controlled factors.
Each evaluation keeps the semantic goal, object set, and success conditions unchanged while controlling two groups of scene factors. Invariance factors alter task-irrelevant visual conditions without changing the intended task behavior; equivariance factors alter task-relevant geometry and require corresponding changes in reaching, grasping, and contact trajectories.
None / Equi. / Inv. use matched anchors aligned by episode index; Full is sampled independently without an anchor.
Results · 26,000 Evaluation Rollouts
Policy Leaderboard
The paper evaluates four visual–proprioceptive baselines across 26 tasks and four channels. π₀.₅ achieves the highest mean stable success rate (27.4%) and leads under the Full shift (26.2%). GR00T N1.5 leads in matched scenes (38.0%) and mean latched stage completion (45.2%). Explore the leaderboard for detailed policy comparisons and complete results.
Paper Baselines
Explore Leaderboard →Four visual–proprioceptive baselines and a tactile GR00T variant. SR and LSCR are percentages; Eff. is success-conditioned efficiency. NR denotes results not reported. Gold outlines mark the best reported values. Explore the leaderboard for task details, evaluation coverage, and complete downloads.
Tactile comparison. GR00T N1.5 with tactile input achieves 25.5% mean SR versus 24.9% without tactile input: 14 tasks improve, 10 decline, and 2 tie. Compare both variants across all 26 tasks →
Get Started
From a scene to an evaluation.
Environment setup, policy training and evaluation, and the full 26-task catalog are in the Bench2Dex Documentation.