BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark
Nikita Cherniadev*, Nicholas Backshall*, Xiao Ma*, Yunfan Lu, Younggyo Seo, Stephen James
BiGym is a new benchmark and learning environment for mobile bi-manual demo-driven robotic manipulation. BiGym features 40 diverse tasks set in home environments, ranging from simple target reaching to complex kitchen cleaning. To capture the real-world performance accurately, we provide human-collected demonstrations for each task, reflecting the diverse modalities found in real-world robot trajectories. BiGym supports a variety of observations, including proprioceptive data and visual inputs such as RGB, and depth from 3 camera views.
For latest updates, check our project page: https://chernyadev.github.io/bigym/
pip install .
Task | Description | Preview |
---|---|---|
ReachTarget | Reach the target with either left or right wrist. | |
ReachTargetSingle | Reach the target with specific wrist. | |
ReachTargetDual | Reach 2 targets, one with each arm. | |
StackBlocks | Move blocks across the table, and stack them in the target area. | |
MovePlate | Move the plate between two draining racks. | |
MoveTwoPlates | Move two plates simultaneously from one draining rack to the other. | |
FlipCup | Flip the cup, initially positioned upside down on the table, to an upright position. | |
FlipCutlery | Take the cutlery from the static holder, flip it, and place it back into the holder. | |
DishwasherOpen | Open the dishwasher door and pull out all trays. | |
DishwasherClose | Push back all trays and close the door of the dishwasher. | |
DishwasherOpenTrays | Pull out the dishwasher’s trays with the door initially open. | |
DishwasherCloseTrays | Push the dishwasher’s trays back with the door initially open. | |
DishwasherLoadPlates | Move plates from the rack to the lower tray of the dishwasher. | |
DishwasherLoadCups | Move cups from the table to the upper tray of the dishwasher. | |
DishwasherLoadCutlery | Move cutlery from the table holder to the dishwasher’s cutlery basket. | |
DishwasherUnloadPlates | Move plates from the tray of the dishwasher to a table rack. | |
DishwasherUnloadCups | Move cups from the upper tray of the dishwasher to the table. | |
DishwasherUnloadCutlery | Move cutlery from the cutlery basket to a tray on the table. | |
DishwasherUnloadPlatesLong | A full task of unloading a plate: picking up the plate from dishwasher, placing this plate into the rack located in the closed wall cabinet, and closing the dishwasher and cupboard. | |
DishwasherUnloadCupsLong | A full task of unloading a cup: picking up the cup, placing it inside the closed wall cabinet, and closing the dishwasher and cupboard. | |
DishwasherUnloadCutleryLong | A full task of unloading a cutlery: picking up a cutlery, placing it into the cutlery tray inside the closed drawer, and closing the dishwasher and drawer. | |
DrawerTopOpen | Open the top drawer of the kitchen cabinet. | |
DrawerTopClose | Close the top drawer of the kitchen cabinet. | |
DrawersAllOpen | Open all sliding drawers of the kitchen cabinet. | |
DrawersAllClose | Close all sliding drawers of the kitchen cabinet. | |
WallCupboardOpen | Open doors of the wall cabinet. | |
WallCupboardClose | Close doors of the wall cabinet. | |
CupboardsOpenAll | Open all drawers and doors of the kitchen set. | |
CupboardsCloseAll | Close all drawers and doors of the kitchen set. | |
PutCups | Pick up cups from the table and put them into the closed wall cabinet. | |
TakeCups | Take two cups out from the closed wall cabinet and put them on the table. | |
PickBox | Pick up a large box from the floor and place it on the counter. | |
StoreBox | Move a large box from the counter to the shelf in the cabinet below. | |
SaucepanToHob | Take the saucepan from the closed cabinet and place it on the hob. | |
StoreKitchenware | Take all items from the hob and place them in the cabinet below. | |
ToastSandwich | Use the spatula to put the sandwich on the frying pan. | |
FlipSandwich | Flip the sandwich in the frying pan using the spatula. | |
RemoveSandwich | Take the sandwich out of the frying pan. | |
GroceriesStoreLower | Place a random set of groceries in the cabinets below the counter. | |
GroceriesStoreUpper | Place a random set of groceries in cabinets and shelves on the wall. |
Directly instantiate the task of interest. Tasks are located in bigym/envs/.
from bigym.action_modes import TorqueActionMode
from bigym.envs.reach_target import ReachTarget
from bigym.utils.observation_config import ObservationConfig, CameraConfig
env = ReachTarget(
action_mode=TorqueActionMode(floating_base=True),
observation_config=ObservationConfig(
cameras=[
CameraConfig(
name="head",
rgb=True,
depth=False,
resolution=(128, 128),
)
],
),
render_mode=None,
)
Use ActionModes
to parameterise how you want to control your robot.
Please see simple example here: examples/replay_demo.py.
Demonstrations are automatically downloaded from GitHub releases.
When demos are requested by calling DemoStore.get_demos()
. The current dataset will be cached locally at $HOME/.bigym/v0.0.0
.
Demonstrations with custom observations and frequency are also cached to $HOME/.bigym/v0.0.0
.
JointPositionActionMode(floating_base=True, absolute=True)
JointPositionActionMode(floating_base=True, absolute=False)
Replay existing demos using GUI player.
python tools/demo_player/main.py
Record new demos in VR. Follow VR README to configure docker container to run this tool.
python tools/demo_recorder/main.py
On each PR, please ensure to bump the:
- Major version if you alter the existing interface in any way.
- Minor version if you have added new features (which didn't break the existing interface)
- Patch version for bug fixes.
Please ensure that you pass pre-commits before opening a PR: pre-commit run --all-files
and that you pass all tests: pytest tests/ --run-slow
.
- BiGym License (Apache 2.0) - This repository
- Mujoco Menagerie (Apache 2.0) - Models of robots and grippers
- 3D Assets Attributions (CC0, CC BY 4.0, CC BY NC 4.0) - 3D Assets
If you find our work helpful, please kindly cite us
@article{chernyadev2024bigym,
title={BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark},
author={Chernyadev, Nikita and Backshall, Nicholas and Ma, Xiao and Lu, Yunfan and Seo, Younggyo and James, Stephen},
journal={arXiv preprint arXiv:2407.07788},
year={2024}
}