Extending the Framework

For advanced users seeking deeper customization, this chapter demonstrates how to extend RLinf by integrating custom environments and new model architectures.

You will learn how to:

• Integrate a new environment into RLinf’s task system

• Add a new model using the FSDP + HuggingFace backend

• Add a new model using the Megatron + SGLang backend

RLinf supports multiple backends for model training, each with its own initialization logic and execution flow. This guide provides step-by-step instructions on how to:

• Register and load custom models in RLinf

• Configure YAML files to reference your new model or environment

• Extend backend-specific code if your model type is not yet supported

• Adapt environment wrappers and interfaces to integrate new simulators or APIs

Whether you’re training a novel model architecture or experimenting with a custom RL environment, this section gives you the tools to plug directly into RLinf’s modular design.