RL with RoboVerse Benchmark#

https://roboverseorg.github.io/static/images/teaser.jpg — RoboVerse (image: RoboVerse).#

RoboVerse is a simulator suite for robot manipulation tasks across multiple backends. You’ll use RLinf to PPO-fine-tune an OpenPI π₀.₅ policy on a RoboVerse kitchen manipulation task.

Overview#

Fine-tune OpenPI π₀.₅ on a RoboVerse task with two RGB views and sparse rewards.

Models

π₀.₅

Algorithms

PPO

Tasks

Bowl on cabinet

Hardware

1 node · 4 GPUs

You’ll do: install → download resources + model → launch run_embodiment.sh → watch env/success_once.

Prerequisites: Installation · RoboVerse resources · an SFT checkpoint.

Tasks#

Task	Description
`libero_90.kitchen_scene1_put_the_black_bowl_on_top_of_the_cabinet`	Put the black bowl on top of the cabinet in a kitchen scene.

Observation and Action#

Field	Specification
Observation	Main camera RGB and wrist-camera RGB at 224×224 plus an 8-dim proprioceptive state.
Action	7-dim continuous action: 3D end-effector position, 3D rotation vector, and gripper.
Reward	Sparse task-completion reward.
Prompt	Natural-language instruction for the RoboVerse task.

Installation#

First, clone the RLinf repository:

# Mainland China users can use a mirror for faster cloning:
# git clone https://ghfast.top/github.com/RLinf/RLinf.git
git clone https://github.com/RLinf/RLinf.git
cd RLinf

Then set up the dependencies with one of the two methods below — a prebuilt Docker image (recommended) or a custom environment. The general setup (prerequisites, GPU drivers, the in-image switch_env helper, mirrors, and troubleshooting) is documented once in Installation; the commands in this recipe only differ in the Docker image tag and the --env value.

Docker image

docker run -it --rm --gpus all \
   --shm-size 32g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-roboverse

# For mainland China users:
# docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-roboverse

Switch to the OpenPI virtual environment inside the image:

source switch_env openpi

Custom environment

Install RoboVerse with the OpenPI dependencies:

# Mainland China users can add --use-mirror.
bash requirements/install.sh embodied --model openpi --env roboverse
source .venv/bin/activate

Download the default RoboVerse resources:

cd /path/to/RLinf
# export HF_ENDPOINT=https://hf-mirror.com
hf download --repo-type dataset manity/roboverse_data --local-dir .

Download the Model#

Download the OpenPI π₀.₅ checkpoint used by the reference config:

cd /path/to/save/model

git lfs install
git clone https://huggingface.co/RLinf/RLinf-Pi05-LIBERO-SFT

# Or use huggingface-hub:
# export HF_ENDPOINT=https://hf-mirror.com
pip install huggingface-hub
hf download RLinf/RLinf-Pi05-LIBERO-SFT --local-dir RLinf-Pi05-LIBERO-SFT

After downloading, point your config YAML at the checkpoint — set the same path for both the rollout and the actor model:

rollout:
   model:
      model_path: /path/to/downloaded-checkpoint
actor:
   model:
      model_path: /path/to/downloaded-checkpoint

Run It#

Launch the RoboVerse recipe:

Recipe	Config	Command suffix
OpenPI π₀.₅ + PPO	`examples/embodiment/config/roboverse_ppo_openpi_pi05.yaml`	`roboverse_ppo_openpi_pi05`

bash examples/embodiment/run_embodiment.sh roboverse_ppo_openpi_pi05

What this does:

Starts the embodied training entrypoint with the RoboVerse Hydra config.
Creates Ray workers for the actor, rollout, and RoboVerse env components.
Runs PPO rollouts, computes sparse task rewards, and updates the OpenPI policy.

For standalone evaluation, use the unified Evaluation CLI with config fallback and the same suffix, roboverse_ppo_openpi_pi05.

Note

The default config places actor and rollout on GPUs 0-1 and env workers on GPUs 2-3. Tune cluster.component_placement, env.train.total_num_envs, and actor.global_batch_size for your hardware.

Visualization and Results#

Launch TensorBoard from the RLinf repo root:

tensorboard --logdir ../results --port 6006

The key signal is env/success_once. For every logged metric, see Training metrics.

Enable video in the env config when you want rollout videos:

env:
  eval:
    video_cfg:
      save_video: True
      video_base_dir: ${runner.logger.log_path}/video/eval

Enable W&B or SwanLab by adding logger backends:

runner:
  logger:
    logger_backends: ["tensorboard", "wandb"]  # or swanlab

Note

This page does not publish a fixed RoboVerse success-rate table yet. Use env/success_once and evaluation videos to compare your runs.