RunPod

RunPod is an end-to-end AI cloud that simplifies building and deploying models. You can easily launch a GPU pod in seconds and run training jobs or inference workloads.

Use the following steps to run SkyRL on Runpod. You can customize them to your needs. These instructions were tested end-to-end on a single A40 GPU on Runpod, training on the GSM8K dataset. It's a quick, minimal path to your first training run.

First, install Miniconda:

cd $HOME
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm ~/miniconda3/miniconda.sh
source ~/miniconda3/bin/activate
conda init --all

Close the terminal and reopen it to activate the base conda environment. Then run the following snippet. The NUMA installation follows the system dependencies guide.

cd $HOME
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r

# Optionally, export your WANDB_API_KEY
echo "export WANDB_API_KEY=YOUR_WANDB_API_KEY" >> ~/.bashrc

# Sometimes Runpod places `.cache` under `/workspace`, which can be slow.
# We set them under `~/` here.
mkdir -p "$HOME/.cache"
echo 'export UV_CACHE_DIR="$HOME/.cache/uv"' >> ~/.bashrc
echo 'export HF_HOME="$HOME/.cache/huggingface"' >> ~/.bashrc

# ---------------
# Install numactl (libnuma)
# ---------------
# Get the source
cd $HOME
wget https://github.com/numactl/numactl/releases/download/v2.0.16/numactl-2.0.16.tar.gz
tar xzf numactl-2.0.16.tar.gz
cd numactl-2.0.16

# Build to a local prefix
./configure --prefix=$HOME/.local
make
make install
cd $HOME

# Point the compiler and linker to it
echo "export CPATH=$HOME/.local/include:$CPATH" >> ~/.bashrc
echo "export LIBRARY_PATH=$HOME/.local/lib:$LIBRARY_PATH" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=$HOME/.local/lib:$LD_LIBRARY_PATH" >> ~/.bashrc

Close the terminal and reopen it. Then launch a basic GSM8K training run with the following commands. For more, see quickstart.

cd $HOME
git clone https://github.com/NovaSky-AI/SkyRL
cd SkyRL/
pip install uv
uv run --isolated python examples/train/gsm8k/gsm8k_dataset.py --output_dir $HOME/data/gsm8k
NUM_GPUS=1 LOGGER=console bash examples/train/gsm8k/run_gsm8k.sh