Extending#

Choose the smallest extension point that matches your change. If RLinf already connects the physical hardware, add a real-world task without touching robotics. If you need a new sensor, actuator, or robot, start with one local part and add placement later. For model work, continue from FSDP to Megatron, SGLang, and SFT, then use Advanced Integrations for backend integration and weight transport.

Extending Overview

Where each extension point lives and how the pieces fit together.

Extending the Framework
New Environment

Add a new RL environment and wire it into the env registry.

Adding New Environment
New Real-World Tasks

Add a task on physical hardware RLinf already supports.

New Real-World Tasks
New Robot

Add one local part, compose the robot, then place it remotely.

Adding a Robot
New Model with FSDP

Add a HuggingFace model on the FSDP backend.

Adding New Models with FSDP+HuggingFace
New Model with Megatron

Add a HuggingFace model on the Megatron+SGLang backend.

Adding New Models with Megatron+SGLang
SGLang Embodied Model

Adapt the embodied model to the RLinf rollout worker using the sglang backend, and use various simulators to evaluate the model.

Adapting an Embodied Model in SGLang Server
New SFT Model

Plug a new model into the SFT training flow.

Add a New Model for SFT Training with FSDP + HuggingFace
Advanced Integrations

Add Megatron-Bridge and weight synchronization workflows.

Advanced Integrations