RL with Genesis Benchmark

Genesis (image: Genesis).

Genesis is a GPU-accelerated multi-physics simulator for robotics. You’ll use RLinf to train MLP or CNN policies with PPO on a Franka cube-pick task.

Overview

Train a Franka Panda policy to pick up a cube in Genesis.

Models

MLP · CNN

Algorithms

PPO

Tasks

CubePick

Hardware

1 node · 1 GPU

You’ll do: install → optionally download ResNet → launch run_embodiment.sh → watch env/success_once.

Prerequisites: Installation · Genesis dependencies from the install step.

Tasks

Task	Description
cube_pick	Control a Franka Panda arm to grasp and lift a cube.

Observation and Action

Field	Specification
Observation	16-dim state for MLP; 256×256 RGB plus 16-dim state for CNN.
Action	9-dim continuous action: 7 Franka arm joint positions plus 2 gripper positions.
Reward	Dense approach reward plus grasp-success bonus.
Prompt	Not used; this is a low-dimensional/CNN policy-control recipe.

Installation

First, clone the RLinf repository:

# Mainland China users can use a mirror for faster cloning:
# git clone https://ghfast.top/github.com/RLinf/RLinf.git
git clone https://github.com/RLinf/RLinf.git
cd RLinf

Then set up the dependencies with one of the two methods below — a prebuilt Docker image (recommended) or a custom environment. The general setup (prerequisites, GPU drivers, the in-image switch_env helper, mirrors, and troubleshooting) is documented once in Installation; the commands in this recipe only differ in the Docker image tag and the --env value.

Docker image

docker run -it --rm --gpus all \
   --shm-size 32g \
   --network host \
   --name rlinf \
   -v .:/workspace/RLinf \
   rlinf/rlinf:agentic-rlinf0.3-genesis

# For mainland China users:
# docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-genesis

Custom environment

Install Genesis dependencies:

# Mainland China users can add --use-mirror.
bash requirements/install.sh embodied --env genesis
source .venv/bin/activate

Download the Model

Skip this section for the MLP + PPO recipe. For the CNN + PPO recipe, download the ResNet checkpoint:

cd /path/to/save/model

git lfs install
git clone https://huggingface.co/RLinf/RLinf-ResNet10-pretrained

# Or use huggingface-hub:
# export HF_ENDPOINT=https://hf-mirror.com
pip install huggingface-hub
hf download RLinf/RLinf-ResNet10-pretrained --local-dir RLinf-ResNet10-pretrained

Then set the same checkpoint path for rollout and actor in examples/embodiment/config/genesis_cubepick_ppo_cnn.yaml:

rollout:
   model:
      model_path: /path/to/RLinf-ResNet10-pretrained
actor:
   model:
      model_path: /path/to/RLinf-ResNet10-pretrained

Run It

Pick one recipe and launch training:

Recipe	Config	Command suffix
MLP + PPO	examples/embodiment/config/genesis_cubepick_ppo_mlp.yaml	genesis_cubepick_ppo_mlp
CNN + PPO	examples/embodiment/config/genesis_cubepick_ppo_cnn.yaml	genesis_cubepick_ppo_cnn

bash examples/embodiment/run_embodiment.sh genesis_cubepick_ppo_mlp
bash examples/embodiment/run_embodiment.sh genesis_cubepick_ppo_cnn

What this does:

1. Starts the embodied training entrypoint with the selected Hydra config.

2. Creates Ray workers for the actor, rollout, and Genesis env components.

3. Runs PPO rollouts, computes cube-pick rewards, and updates the selected policy.

Note

Both configs run on GPU 0 by default. Tune cluster.component_placement, env.train.total_num_envs, and batch sizes for your hardware.

Visualization and Results

Launch TensorBoard from the RLinf repo root:

tensorboard --logdir ../results --port 6006

The key signal is env/success_once. For every logged metric, see Training metrics.

Enable video in the env config when needed:

env:
  eval:
    video_cfg:
      save_video: True
      video_base_dir: ${runner.logger.log_path}/video/eval

Recipe	Reported Behavior
MLP + PPO	With the default genesis_cubepick_ppo_mlp parameters, env/success_once reaches about 80%.