RL with Real-World Robots

Use this section when your starting point is physical robot hardware. Start with Franka if you use a Franka arm or a Franka-based rig; use the other robot pages for GimArm, XSquare Turtle2, and Dexmal DOS-W1.

Each section gives the setup path for teleoperation, data collection, sim-to-real transfer, deployment, or online RL.

Franka
Use the Franka section for base real-world RL, reward models, ZED + Robotiq, GELLO, dual-arm rigs, dexterous hands, Pi0 SFT, and HG-DAgger

GimArm
Train a 6-DOF GimArm peg-insertion task over SocketCAN with Pinocchio-based FK

XSquare Turtle2
Run SAC with a CNN policy on the XSquare Turtle2 dual-arm robot

Dexmal DOS-W1
Train a flow-matching + SAC pick task on the Dexmal DOS-W1 dual-arm robot