RL with Real-World Robots#
Use this section when your starting point is physical robot hardware. Start with Franka if you use a Franka arm or a Franka-based rig; use the other robot pages for ARX X5, YAM, GimArm, XSquare Turtle2, Dexmal DOS-W1, AgileX Piper, SO101, and LeKiwi.
Choose a guide for hardware checks, teleoperation, data collection, sim-to-real transfer, deployment, or online RL.
Single-Arm Franka
Use the single-arm Franka section for real-world RL, reward models, data collection, dexterous hands, SFT, and deployment
Dual-Arm Franka
Use the dual-arm Franka section for collection, SFT, deployment, and PICO-assisted HG-DAgger
GimArm
Train a 6-DOF GimArm peg-insertion task over SocketCAN with Pinocchio-based FK
XSquare Turtle2
Run SAC with a CNN policy on the XSquare Turtle2 dual-arm robot
Dexmal DOS-W1
Train a flow-matching + SAC pick task on the Dexmal DOS-W1 dual-arm robot
ARX X5
Collect dual-arm demonstrations with PICO controllers and three RGB cameras
YAM#
Use the YAM section for dual-arm setup and data collection guides.
Piper, SO101, and LeKiwi Setup#
Use these guides to connect a robot and run its hardware test script. Piper, SO101, and LeKiwi do not yet have supported real-world training workflows.
Configure CAN and run the Piper joint and gripper test script.
Set up motors, calibrate an SO-101, and run the joint and gripper test script.
Set up the wheel motors, check the front webcam, and drive the mobile base.