Environment Variables

module env_vars

Environment variable configuration for SkyRL.

All environment variables used by SkyRL should be defined here for discoverability.

Attributes:

attr SKYRL_RAY_PG_TIMEOUT_IN_S

SKYRL_RAY_PG_TIMEOUT_IN_S = int(os.environ.get('SKYRL_RAY_PG_TIMEOUT_IN_S', 180))

Timeout for allocating the placement group for different actors in SkyRL.

attr SKYRL_WORKER_NCCL_TIMEOUT_IN_S

SKYRL_WORKER_NCCL_TIMEOUT_IN_S = int(os.environ.get('SKYRL_WORKER_NCCL_TIMEOUT_IN_S', 600))

Timeout for initializing the NCCL process group for the worker, defaults to 10 minutes.

attr SKYRL_VLLM_DP_PORT_OFFSET

SKYRL_VLLM_DP_PORT_OFFSET = int(os.environ.get('SKYRL_VLLM_DP_PORT_OFFSET', 500))

Offset for the data parallel port of the vLLM server.

attr SKYRL_WAIT_UNTIL_INFERENCE_SERVER_HEALTHY_TIMEOUT_S

SKYRL_WAIT_UNTIL_INFERENCE_SERVER_HEALTHY_TIMEOUT_S = int(os.environ.get('SKYRL_WAIT_UNTIL_INFERENCE_SERVER_HEALTHY_TIMEOUT_S', 600))

Timeout for waiting until the inference server is healthy.

attr SKYRL_INFERENCE_CONTROL_PLANE_TIMEOUT_S

SKYRL_INFERENCE_CONTROL_PLANE_TIMEOUT_S = float(os.environ.get('SKYRL_INFERENCE_CONTROL_PLANE_TIMEOUT_S', 600))

Per-request timeout for control-plane calls from RemoteInferenceClient to the inference servers (pause, resume, sleep, wake_up, weight updates, LoRA load/unload, ...). Defaults to SKYRL_WORKER_NCCL_TIMEOUT_IN_S's default. <= 0 disables it. Data-plane requests (generate, chat completions, ...) are not bounded.

attr SKYRL_HTTP_CONNECTION_LIMIT

SKYRL_HTTP_CONNECTION_LIMIT = int(os.environ.get('SKYRL_HTTP_CONNECTION_LIMIT', 50000))

Maximum number of concurrent HTTP connections for the inference client, router, and server.

This controls:

• aiohttp TCPConnector limit in RemoteInferenceClient
• connection pool limits in the router
• uvicorn TCP backlog in the router and vLLM server

attr SKYRL_GENERATE_CONCURRENCY_PER_ENGINE

SKYRL_GENERATE_CONCURRENCY_PER_ENGINE = int(os.environ.get('SKYRL_GENERATE_CONCURRENCY_PER_ENGINE', 512))

Maximum number of concurrent generate tasks per inference engine.

The effective concurrency limit is SKYRL_GENERATE_CONCURRENCY_PER_ENGINE * num_engines. Large batch sizes (e.g. 5120) can overwhelm the router's single-threaded event loop and vLLM's accept queue when all requests fire simultaneously. We ensure that at most this many tasks per engine are in-flight at once; the rest queue locally and proceed as slots free up.

Set to 0 to disable throttling (all tasks fire immediately).

attr SKYRL_TINKER_CONTINUOUS_SAMPLING

SKYRL_TINKER_CONTINUOUS_SAMPLING = str(os.environ.get('SKYRL_TINKER_CONTINUOUS_SAMPLING', 'True')).lower() in ('true', '1', 'yes')

Whether the Tinker engine completes each sample future as soon as its own generation finishes (continuous sampling) instead of batching pending sample requests and completing them together once the whole batch is done.

Only takes effect with backends that expose the per-request async sample path (the skyrl-train backends); other backends always use the batched loop.

Default: True. Set SKYRL_TINKER_CONTINUOUS_SAMPLING=0 to use the batched loop.

attr SKYRL_LD_LIBRARY_PATH_EXPORT

SKYRL_LD_LIBRARY_PATH_EXPORT = str(os.environ.get('SKYRL_LD_LIBRARY_PATH_EXPORT', 'False')).lower() in ('true', '1', 'yes')

Whether to export LD_LIBRARY_PATH environment variable from the driver to the workers with Ray's runtime env.

For example, if you are using RDMA, you may need to customize the LD_LIBRARY_PATH to include the RDMA libraries (Ex: EFA on AWS).

attr SKYRL_PYTHONPATH_EXPORT

SKYRL_PYTHONPATH_EXPORT = str(os.environ.get('SKYRL_PYTHONPATH_EXPORT', 'False')).lower() in ('true', '1', 'yes')

Whether to export PYTHONPATH environment variable from the driver to the workers with Ray's runtime env.

See https://github.com/ray-project/ray/issues/56697 for details on why this is needed.

attr SKYRL_DISABLE_FA4

SKYRL_DISABLE_FA4 = str(os.environ.get('SKYRL_DISABLE_FA4', 'False')).lower() in ('true', '1', 'yes')

Force Transformer Engine to ignore FlashAttention 4, falling back to FA2 (or cuDNN fused attention). FA4 is opt-in via the fa4 extra -- SkyRL ships FA4's kernels inside the combined flash-attn wheel, but TE only enables them when the metadata-only flash-attn-4 companion is installed. This variable turns FA4 back off for an environment that already has it, without re-resolving.

Useful for A/B-ing FA2 against FA4 without rebuilding the environment, and as an escape hatch if an FA4 kernel misbehaves on a shape SkyRL exercises.

Default: False (use FA4 where supported).

attr SKYRL_DUMP_INFRA_LOG_TO_STDOUT

SKYRL_DUMP_INFRA_LOG_TO_STDOUT = str(os.environ.get('SKYRL_DUMP_INFRA_LOG_TO_STDOUT', 'False')).lower() in ('true', '1', 'yes')

When enabled, infrastructure logs (vLLM, Ray, workers) are shown on stdout instead of being redirected to the log file. Useful for debugging startup issues.

Default: False (infrastructure logs go to file only, stdout shows training progress). Set SKYRL_DUMP_INFRA_LOG_TO_STDOUT=1 to show all logs on stdout.

attr SKYRL_FORWARDING_INFERENCE_TIMEOUT_SEC

SKYRL_FORWARDING_INFERENCE_TIMEOUT_SEC = float(os.environ.get('SKYRL_FORWARDING_INFERENCE_TIMEOUT_SEC', 300))

Read timeout in seconds for API-side requests forwarded to the SkyRL-Train-managed inference engine. This is applicable for SkyRL's Tinker server in non-colocated setups.

The timeout must cover time spent queued behind other requests as well as generation time. Equivalent to the --forwarding-inference-timeout-sec flag of the Tinker API server (EngineConfig.forwarding_inference_timeout_sec); the flag takes precedence when both are set.

attr SKYRL_MEGATRON_RANDOM_INIT

SKYRL_MEGATRON_RANDOM_INIT = str(os.environ.get('SKYRL_MEGATRON_RANDOM_INIT', '0')).lower() in ('1', 'true')

Build the Megatron model from the HF config with random weights instead of loading the checkpoint.

For throughput/memory benchmarking (e.g. dummy SFT with random tokens), where only the architecture matters: model.path then needs just config.json and tokenizer files, no safetensors.