Limitations & Roadmap
This page documents the features that are not yet wired through the Tinker integration. Everything listed on the Overview as supported is expected to work; if you hit an issue with one of those paths, please open an issue.
Current Limitations
Multi-tenant LoRA: Megatron only
Multi-tenant LoRA training and sampling are supported on the Megatron backend with vLLM serving per-tenant adapters by name. See Multi-tenancy for the operator contract and SL/RL quickstarts. FSDP support is pending, and full-parameter fine-tuning remains single-tenant on both backends — calling create_model with lora_rank=0 while another model exists returns an error.
All adapters registered against one server must share the same (rank, alpha, train_attn, train_mlp) signature; mismatched signatures are hard-rejected at create_model. seed is not part of it — it changes an adapter's initial values, not its shape. On the skyrl-train backend seed is not applied at all: LoRA initialization follows the server's trainer.seed. target_modules is fixed server-side and so is identical across adapters by construction.
LoRA group flags
train_attn and train_mlp are honoured on both skyrl-train backends, where they narrow target_modules="all-linear" to the attention or MLP projections. The exported adapter records which modules it covers, asserted in tests/tinker/skyrl_train/test_megatron_lora_flags.py and tests/tinker/skyrl_train/test_fsdp_lora_flags.py. All tenants on one server share the first tenant's flags (see the signature above).
- LoRA is not applied to the unembedding layer, so
train_unembedis ignored. This follows common LoRA practice: PEFT's"all-linear"leaves the output layer out too. Adaptinglm_headwould need separate handling that depends on the architecture, since many models tie it to the input embeddings. The Tinker SDK defaultstrain_unembedtoTrue, so the server accepts the flag rather than rejecting every client: it logs a warning and reports the model's config back withtrain_unembed=False.
At least one of train_attn / train_mlp must be True: an adapter covering no modules would have nothing to train. Narrowing an explicit target_modules list is rejected; select the modules in the list instead. Narrowing is also rejected with trainer.policy.model.lora.init_adapter_path, since the adapter it loads decides which modules are adapted.
Megatron
Megatron expands "all-linear" to Megatron-Core module names, which are the same for every architecture it supports, and narrowing keeps one half of that list:
| Group | lora_type: lora | lora_type: canonical_lora |
|---|---|---|
| Attention | linear_qkv, linear_proj, in_proj, out_proj | linear_q, linear_k, linear_v, linear_proj, in_proj, out_proj |
| MLP | linear_fc1, linear_fc2 | linear_fc1_up, linear_fc1_gate, linear_fc2 |
The MLP group covers dense MLP layers, shared experts and routed experts. in_proj / out_proj belong to Gated DeltaNet layers (Qwen3.5, Qwen3-Next) and Mamba layers (Nemotron-3), so both count as attention. Two cases raise while the model is built:
- Vision-language models with a vision tower. The tower and its patch merger reuse the language model's module names, so either half would adapt them too.
trainer.policy.language_model_only=truedrops the tower for Qwen3.5 and Kimi K2.5. - Attention-only on multi-latent attention models (DeepSeek-V3, Kimi K2, GLM-4.7-Flash, GLM-5.3-Flash).
"all-linear"reaches onlylinear_projof their attention. MLP-only works.
On the first create_model these errors come after GPUs are reserved, and the server must be restarted before it accepts another model (#2482). Leaving both flags True is unaffected.
FSDP
Narrowing needs Llama/Qwen-style module names. The flags select layers by their parent module: self_attn (or linear_attn) for attention, mlp (or experts) for the MLP. Llama, Qwen, Gemma, Mistral and Phi-3 follow this. Architectures that use other names — GPT-NeoX, Falcon and BLOOM (attention / self_attention), Llama 4 (feed_forward), GPT-2 (Conv1D layers) — fail with No attn linear layers found (or mlp) when the matching group is requested. Narrowing also raises when the chosen group shares a layer name with a layer outside it, as GPT-BigCode's MLP and attention both have a c_proj. Leaving both flags True is unaffected. For those models, name the modules directly in trainer.policy.model.lora.target_modules.
Narrowing is not supported for vision-language models. Their vision towers and projectors reuse layer names from the language model, so a narrowed adapter would also cover modules outside the requested group. Requesting it raises; leaving both flags True is unaffected.
Routed MoE experts are not covered on FSDP. Current transformers stores routed experts as fused 3-D nn.Parameter tensors rather than nn.Linear modules, and PEFT can only adapt the latter. On a model with only routed experts (Qwen3-MoE, Mixtral) the MLP group is therefore empty and narrowing to it raises. On a model that also has shared experts or dense MLP layers (DeepSeek-V3, Qwen2-MoE), narrowing to the MLP group adapts those and leaves the routed experts untouched. Megatron adapts routed experts in the MLP group.
No Prompt Logprobs
The sample() API does not yet return prompt logprobs, even when requested. A warning is logged but no error is raised. This may affect scripts that rely on prompt logprobs for KL penalty computation.
KL Penalty
KL penalty (kl_penalty_coef > 0) is not yet supported. This requires prompt logprobs from vLLM (not wired yet) and a way to serve frozen base model logprobs after weight sync. This is disabled by default in cookbook recipes, so it is not a blocker for most workflows.
RL Loss Functions
Only cross_entropy and importance_sampling are currently wired through the Tinker data conversion path. SkyRL's PolicyLossRegistry contains implementations for PPO (regular), cispo, and others, but these are not yet validated through the Tinker API.