Configuration

This page describes how to configure the SkyRL Tinker backend, including GPU allocation, training parameters, and inference settings.

When spinning up the Tinker server, the --backend-config flag accepts a JSON dictionary of dot-notation overrides that are applied to the underlying SkyRL-Train configuration. For example:

uv run --extra tinker --extra fsdp -m skyrl.tinker.api \
    --base-model "Qwen/Qwen3-4B-Instruct-2507" --backend fsdp \
    --backend-config '{"trainer.placement.policy_num_gpus_per_node": 4, "generator.inference_engine.num_engines": 4}'

Any field in the SkyRL-Train config can be overridden this way (see SkyRLTrainConfig in skyrl/train/config/config.py for all available keys and defaults), except the keys the server sets itself (see Keys the server sets). The most commonly used options are listed below.

JSON types are preserved exactly, so use JSON literals rather than strings: null for an unset field, true/false for booleans, and 4 rather than "4" for numbers.

GPU and Parallelism

When running a small model on multiple GPUs, you typically want to set policy_num_gpus_per_node and generator.inference_engine.num_engines to the same value. For example, on a 4-GPU node:

--backend-config '{"trainer.placement.policy_num_gpus_per_node": 4, "generator.inference_engine.num_engines": 4}'

For large models that don't fit on a single GPU for inference, increase generator.inference_engine.tensor_parallel_size and decrease generator.inference_engine.num_engines accordingly. For example, on 4 GPUs with TP=2:

--backend-config '{"trainer.placement.policy_num_gpus_per_node": 4, "generator.inference_engine.num_engines": 2, "generator.inference_engine.tensor_parallel_size": 2}'

Keys the server sets

The server sets some SkyRL-Train config values itself, from its flags or from each client request. Setting one of these keys in --backend-config to a value the server would not use is an error that lists each offending key. Nested spellings such as {"trainer.policy.optimizer_config": {"lr": 1e-5}} are checked the same way.

• strategy, runtime_role and served_model_name stop the server at startup.
• The other keys are checked when the first create_model request sets up training (on an inference-only runtime, at the first sample request). That request fails with the error, which also appears in the server log.
• A backend_config LoRA alpha, for the policy or the critic, is used as given: the Tinker SDK cannot send alpha, so the request only carries a placeholder (32).
• Values that SkyRL-Train itself adjusts while building the config are kept, and SkyRL-Train logs a warning for each (for example enforce_eager: true with LoRA on vLLM).

Optimizer settings

Every optim_step applies the full AdamParams from the request: learning_rate, beta1/beta2, eps and weight_decay are written into the optimizer's param groups before the step, and grad_clip_norm becomes the per-step clip (0 disables it, as in the Tinker SDK). On FSDP the request's eps applies too, instead of the previously fixed 1e-8; Megatron's no-decay groups keep their 0. optimizer_config and Megatron's optimizer_config_kwargs are only the build-time defaults, and since the request rewrites them before every step, every AdamParams-shaped key among them is rejected in backend_config (see the table above).

LoRA

LoRA is configured from the client side, not the server. When creating a model via the Tinker SDK, pass a lora_config with the desired rank. For example, in tinker-cookbook recipes:

# LoRA training (default in most recipes)
python -m tinker_cookbook.recipes.sl_loop ... lora_rank=32

# Full-parameter fine-tuning
python -m tinker_cookbook.recipes.sl_loop ... lora_rank=0

No server-side configuration is needed to switch between single-tenant LoRA and full-parameter fine-tuning.

The fields a Tinker request carries — rank, train_attn and train_mlp — come only from the request. Setting trainer.{policy,critic}.model.lora.{rank,train_attn,train_mlp} in backend_config is rejected, since the request would always overwrite it. The exception is an inference-only server (--runtime-role inference), which receives no training requests: there trainer.policy.model.lora.rank in backend_config is how LoRA serving is enabled. Settings the SDK cannot express, such as dropout and target_modules, are configured in backend_config, and so is alpha, for the policy or the critic: the request only carries a placeholder (32).

Multi-tenant LoRA

Hosting multiple LoRA tenants concurrently against one server does require server-side configuration on the Megatron backend. At minimum:

{
    "trainer.placement.colocate_all": false,
    "trainer.policy.megatron_config.lora_config.merge_lora": false,
    "trainer.policy.model.lora.max_loras": <max concurrent adapters in a single batch>,
    "trainer.policy.model.lora.max_cpu_loras": <total adapter capacity>
}

merge_lora: false is required so vLLM serves each tenant's adapter by name (with merge_lora: true vLLM only sees the merged base and per-tenant sampling returns the wrong weights). max_cpu_loras must be sized to the peak number of concurrent tenants — there is no on-demand reload, and if vLLM evicts an adapter the next sample() against it 404s. All adapters on one server must share the same (rank, alpha, train_attn, train_mlp) signature; mismatched signatures are hard-rejected at create_model.

See Multi-tenancy for the full operator contract and SFT/RL quickstarts.

torch.profiler

Traces of policy-worker training steps, driven over plain HTTP on a running server rather than through the Tinker SDK. Start the server with --torch-profiler to enable it; without the flag the endpoints return 404.

uv run --extra tinker --extra fsdp -m skyrl.tinker.api \
    --base-model "Qwen/Qwen3-0.6B" --backend fsdp \
    --torch-profiler '{"export_dir": "/tmp/skyrl_traces", "ranks": [0]}'

Then bracket the steps you care about:

curl -X POST localhost:8000/start_profiling -H 'Content-Type: application/json' -d '{
    "model_id": "<your model id>", "global_step": 120, "export_path_extra": "before-fix",
    "schedule_options": {"skip_first": 0, "wait": 0, "warmup": 1, "active": 5, "repeat": 1}
}'
# ... run training ...
curl -X POST localhost:8000/stop_profiling -H 'Content-Type: application/json' -d '{"model_id": "<your model id>"}'
curl localhost:8000/profiling_status

Traces land in {export_dir}/{global_step}_{export_path_extra}. schedule_options is passed to torch.profiler.schedule and profile_options to torch.profiler.profile. Both are expressed in optim steps: the profiler advances once per optim_step for the requesting model_id, and a trace is written when a window closes, on the transition out of the last active step. /stop_profiling flushes an open window, and for a cloud export_dir blocks until the upload completes.

Only one profiling session runs at a time server-wide — a second /start_profiling gets a 409, and only the owning model_id may stop it. A non-empty target directory is also a 409 unless overwrite: true is passed.

Only policy workers are profiled; inference engines are not. Setting trainer.policy.torch_profiler_config.* in backend_config is rejected (see Keys the server sets) — static config would compete with these endpoints for the same profiler.

See examples/tinker/torch_profiling/ for a runnable client.

Full Config Reference

For the complete list of configuration options, see the SkyRL-Train configuration API reference and the SkyRLTrainConfig dataclass.