Development Guide

Pre-requisities

Follow the installation guide. Make sure that the installation works with our quick start example.

Adding a new environment or task? Follow the new task tutorial. Your custom code can be placed anywhere - building on top of skyrl_train as a package - but we recommend structuring it as a folder similar to examples/multiply.

Creating a custom Generator (ex: porting an agent harness, implementing custom trajectory generation logic, etc.)? Same as the above: your custom code can be placed anywhere and we typically use skyrl-train/examples/ for this. See examples/mini_swe_agent for an example of creating a custom Generator for Mini-SWE-Agent.

Looking to add a new algorithm by changing your advantage estimator or policy loss? Follow the guide for implementing custom algorithms. See examples/algorithms for examples of various custom algorithm implementations.

Looking to modify the training loop for full control? Follow the guide for creating a custom trainer. See examples/algorithms/dapo/main_dapo.py for an example of how to modify the Trainer class for DAPO.

Modifying the existing environment code (ex: adding a custom method for all Env classes, improving the SearchEnv implementation)? You would modify the code in skyrl-gym. Note: you do not have to modify the skyrl-gym package for adding a new environment or task.

Contributing to SkyRL

We welcome all forms of contributions to SkyRL, including bug reports, feature requests, documentation improvements, code contributions etc.

Testing

For code contributions, we recommend running unit tests before submitting a pull request.

SkyRL-train

CPU tests

cd skyrl-train # make sure you are in the correct directory
uv run --isolated --extra dev pytest tests/cpu

GPU tests

The GPU tests require a node with atleast 8 GPUs. They have been tested on a 8xH100 node, but should work even on 8xA100 nodes. We are actively working on making these more accessible.

cd skyrl-train # make sure you are in the correct directory

The tests assume that the GSM8K dataset is downloaded to ~/data/gsm8k. If you have not downloaded the dataset, you can do so by running the following command:

uv run --isolated examples/gsm8k/gsm8k_dataset.py --output_dir ~/data/gsm8k

Finally, you can run the tests by running the following command:

uv run --isolated --extra dev --extra vllm pytest tests/gpu

SkyRL-gym

You can run the tests for the skyrl-gym package by running the following command:

cd skyrl-gym # make sure you are in the correct directory
uv run --isolated --extra dev pytest tests/

Linting and Formatting

We use pre-commit hooks to automatically lint and format the code. You can install the pre-commit hooks by running the following command:

uv pip install pre-commit
pre-commit install

If you wish to manually run linting and formatting:

bash format.sh # run from the root directory

Documentation

We welcome contributions to the documentation. The documentation is stored in the docs/ folder and uses fumadocs.

Developing locally

We recommend building the documentation locally to preview your changes.

cd docs
npm install
npm run dev

This will start the development server at http://localhost:3000.