RL with Real-World Robots#
Use this section when your starting point is physical robot hardware. Start with Franka if you use a Franka arm or a Franka-based rig; use the other robot pages for GimArm, XSquare Turtle2, Dexmal DOS-W1, AgileX Piper, and SO101.
Choose a guide for hardware checks, teleoperation, data collection, sim-to-real transfer, deployment, or online RL.
Single-Arm Franka
Use the single-arm Franka section for real-world RL, reward models, data collection, dexterous hands, SFT, and deployment
Dual-Arm Franka
Use the dual-arm Franka section for collection, SFT, deployment, and PICO-assisted HG-DAgger
GimArm
Train a 6-DOF GimArm peg-insertion task over SocketCAN with Pinocchio-based FK
XSquare Turtle2
Run SAC with a CNN policy on the XSquare Turtle2 dual-arm robot
Dexmal DOS-W1
Train a flow-matching + SAC pick task on the Dexmal DOS-W1 dual-arm robot
Piper and SO101 Setup#
Use these guides to connect an arm and run its hardware test script. Piper and SO101 do not yet have supported real-world tasks or training workflows.
Configure CAN and run the Piper joint and gripper test script.
Set up motors, calibrate an SO-101, and run the joint and gripper test script.