RL with Real-World Robots

Use this section when your starting point is physical robot hardware. Start with Franka if you use a Franka arm or a Franka-based rig; use the other robot pages for GimArm, XSquare Turtle2, and Dexmal DOS-W1.

Each section gives the setup path for teleoperation, data collection, sim-to-real transfer, deployment, or online RL.

Single-Arm Franka
Use the single-arm Franka section for real-world RL, reward models, data collection, dexterous hands, SFT, and deployment

Dual-Arm Franka
Use the dual-arm Franka section for collection, SFT, deployment, and PICO-assisted HG-DAgger

GimArm
Train a 6-DOF GimArm peg-insertion task over SocketCAN with Pinocchio-based FK

XSquare Turtle2
Run SAC with a CNN policy on the XSquare Turtle2 dual-arm robot

Dexmal DOS-W1
Train a flow-matching + SAC pick task on the Dexmal DOS-W1 dual-arm robot