CLI Commands

Use vllm-sr to configure and run the router, manage recipes, send requests, and evaluate routing. See Installation to install the CLI and Local Docker deployment to start a router.

This reference is generated from the registered CLI commands. Command descriptions, arguments, options, and declared defaults match the source. An option without a declared default is omitted until supplied; commands may resolve it from configuration or the environment as described in their help. Repeatable options collect values. Run vllm-sr COMMAND --help for help from your installed version.

Command index

Command	Description
vllm-sr	vLLM Semantic Router CLI - Intelligent routing and caching for vLLM endpoints.
vllm-sr benchmark	Install, run, compare, and gate reproducible benchmark workloads.
vllm-sr benchmark benchmark-install	Install a data-only Benchmark Pack for replay or supported live tracks.
vllm-sr benchmark benchmarks	Print all exact-pinned external benchmark adapter descriptors.
vllm-sr benchmark catalog	Print the versioned evaluation suites, tracks, and targets.
vllm-sr benchmark compare	Compare two local immutable drafts without rerunning either workload.
vllm-sr benchmark gate	Emit standalone gate evidence and fail CI on blocking draft verdicts.
vllm-sr benchmark intelligence	Plan and run the six fixed Intelligence 1.0 benchmarks.
vllm-sr benchmark intelligence list	Print the exact Intelligence 1.0 benchmark and source contracts.
vllm-sr benchmark intelligence plan	Emit secret-free commands and provenance without executing a benchmark.
vllm-sr benchmark intelligence run	Execute fixed benchmark adapters and write private evidence receipts.
vllm-sr benchmark normalizers	Print the closed native-export contracts and explicit blockers.
vllm-sr benchmark report	Read the local worker report draft.
vllm-sr benchmark run	Execute a manifest and print its finalized worker report draft.
vllm-sr benchmark suite-install	Install a normalized suite as exploratory E0 evidence.
vllm-sr benchmark suite-list	List browser-safe suite metadata without private artifact references.
vllm-sr benchmark suite-normalize	Parse one native export into exploratory E0 replay evidence.
vllm-sr benchmark suite-show	Print the immutable operator manifest for one installed suite.
vllm-sr benchmark validate	Validate a fixed evaluation manifest without executing it.
vllm-sr benchmark verify-source	Verify an ignored external source checkout against its exact pin.
vllm-sr completion	Generate or install shell completion for vllm-sr.
vllm-sr completion install	Install shell completions into your shell configuration.
vllm-sr completion show	Print the completion script for a shell.
vllm-sr config	Print generated configuration or run config subcommands.
vllm-sr config apply	Plan, compare-and-swap, persist, and hot-reload a configuration.
vllm-sr config envoy	Print the generated Envoy configuration.
vllm-sr config get	Read the active canonical configuration from a Router.
vllm-sr config import	Import a supported external config source into canonical v0.3 YAML.
vllm-sr config init	Create a minimal canonical configuration template.
vllm-sr config migrate	Migrate a legacy or mixed config file to canonical v0.3 YAML.
vllm-sr config plan	Validate and plan an exact remote mutation without changing the Router.
vllm-sr config rollback	Compare-and-swap the active configuration to a backup version.
vllm-sr config router	Print the canonical router configuration.
vllm-sr config schema	Discover the canonical config contract progressively.
vllm-sr config validate	Validate configuration file.
vllm-sr config versions	List immutable configuration backup versions.
vllm-sr dashboard	Open the dashboard in your default web browser.
vllm-sr logs	Show logs from vLLM Semantic Router service.
vllm-sr optimize	Analyze routing evidence and produce candidate recipe changes.
vllm-sr optimize recipe-learning	Analyze replay and outcomes to produce recipe-learning artifacts.
vllm-sr recipe	Validate, plan, apply, inspect, or package routing recipes.
vllm-sr recipe apply	Validate then compare-and-swap one recipe into the active config.
vllm-sr recipe builtin	Discover, export, or bind installed Recipes without a running Router.
vllm-sr recipe builtin export	Export an exact five-file BUNDLE to a new directory.
vllm-sr recipe builtin init	Bind one installed recipe to explicit models and validate the new config.
vllm-sr recipe builtin list	List installed bundles, recipe names, and required candidate pools as JSON.
vllm-sr recipe delete	Compare-and-swap deletion of an unreferenced named recipe.
vllm-sr recipe get	Read one managed recipe.
vllm-sr recipe list	List recipes and the collection ETag.
vllm-sr recipe pack	Create a deterministic ZIP from an exact five-file RECIPE_DIR.
vllm-sr recipe plan	Validate a recipe and bind the plan to the current config ETag.
vllm-sr recipe validate	Validate a recipe against the running Router without changing config.
vllm-sr request	Send requests through an Envoy listener.
vllm-sr request chat	Send a one-shot chat completion through the Envoy-routed HTTP API.
vllm-sr route	Preview routing decisions or probe the routed inference path.
vllm-sr route preview	Preview signals and the selected route without calling a model backend.
vllm-sr route probe	Probe a real route and assert complete assistant delivery for expected 2xx.
vllm-sr serve	Start vLLM Semantic Router.
vllm-sr status	Show status of vLLM Semantic Router services.
vllm-sr stop	Stop vLLM Semantic Router.
vllm-sr storage	Inspect Router storage or manage local storage credentials.
vllm-sr storage rotate	Replace this stack's storage credentials with freshly generated ones.
vllm-sr storage vector-stores	List vector stores known to the Router management API.

vllm-sr

Usage: vllm-sr [OPTIONS] [COMMAND] [ARGS]...

vLLM Semantic Router CLI - Intelligent routing and caching for vLLM endpoints.

Parameter	Description
--version	Show version and exit. Default: false.
--help	Show this message and exit. Default: false.

vllm-sr benchmark

Usage: vllm-sr benchmark [OPTIONS] COMMAND [ARGS]...

Install, run, compare, and gate reproducible benchmark workloads.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr benchmark benchmark-install

Usage: vllm-sr benchmark benchmark-install [OPTIONS]

Install a data-only Benchmark Pack for replay or supported live tracks.

Parameter	Description
--pack DIRECTORY	Clean exact-revision checkout containing benchmark.yaml and bundle/. [required]
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark benchmarks

Usage: vllm-sr benchmark benchmarks [OPTIONS]

Print all exact-pinned external benchmark adapter descriptors.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr benchmark catalog

Usage: vllm-sr benchmark catalog [OPTIONS]

Print the versioned evaluation suites, tracks, and targets.

Parameter	Description
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark compare

Usage: vllm-sr benchmark compare [OPTIONS]

Compare two local immutable drafts without rerunning either workload.

Parameter	Description
--baseline TEXT	[required]
--candidate TEXT	[required]
--store PATH	[default: .vllm-sr/evaluation-store]
--help	Show this message and exit. Default: false.

vllm-sr benchmark gate

Usage: vllm-sr benchmark gate [OPTIONS] RUN_ID

Emit standalone gate evidence and fail CI on blocking draft verdicts.

Parameter	Description
RUN_ID	Required argument. Type: text.
--store PATH	[default: .vllm-sr/evaluation-store]
--allow-unavailable	Default: false.
--help	Show this message and exit. Default: false.

vllm-sr benchmark intelligence

Usage: vllm-sr benchmark intelligence [OPTIONS] COMMAND [ARGS]...

Plan and run the six fixed Intelligence 1.0 benchmarks.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr benchmark intelligence list

Usage: vllm-sr benchmark intelligence list [OPTIONS]

Print the exact Intelligence 1.0 benchmark and source contracts.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr benchmark intelligence plan

Usage: vllm-sr benchmark intelligence plan [OPTIONS]

Emit secret-free commands and provenance without executing a benchmark.

Parameter	Description
--benchmark TEXT	Exact catalog benchmark ID; repeat it or use all. [default: all] May be repeated.
--model TEXT	[required]
--base-url TEXT	[required]
--source-root DIRECTORY	Directory containing exact-pinned aiperf, inspect-evals, and harbor checkouts. [required]
--output PATH	[required]
--tokenizer TEXT	[default: builtin]
--api-key-env TEXT	[default: OPENAI_API_KEY]
--concurrency INTEGER	[default: 8]
--reasoning-effort TEXT	—
--sample-limit INTEGER	—
--terminal-attempts INTEGER	[default: 5]
--help	Show this message and exit. Default: false.

vllm-sr benchmark intelligence run

Usage: vllm-sr benchmark intelligence run [OPTIONS]

Execute fixed benchmark adapters and write private evidence receipts.

Parameter	Description
--benchmark TEXT	Exact catalog benchmark ID; repeat it or use all. [default: all] May be repeated.
--model TEXT	[required]
--base-url TEXT	[required]
--source-root DIRECTORY	Directory containing exact-pinned aiperf, inspect-evals, and harbor checkouts. [required]
--output PATH	[required]
--tokenizer TEXT	[default: builtin]
--api-key-env TEXT	[default: OPENAI_API_KEY]
--concurrency INTEGER	[default: 8]
--reasoning-effort TEXT	—
--sample-limit INTEGER	—
--terminal-attempts INTEGER	[default: 5]
--help	Show this message and exit. Default: false.

vllm-sr benchmark normalizers

Usage: vllm-sr benchmark normalizers [OPTIONS]

Print the closed native-export contracts and explicit blockers.

Parameter	Description
--runnable-only	Default: false.
--help	Show this message and exit. Default: false.

vllm-sr benchmark report

Usage: vllm-sr benchmark report [OPTIONS] RUN_ID

Read the local worker report draft.

Parameter	Description
RUN_ID	Required argument. Type: text.
--store PATH	[default: .vllm-sr/evaluation-store]
--help	Show this message and exit. Default: false.

vllm-sr benchmark run

Usage: vllm-sr benchmark run [OPTIONS]

Execute a manifest and print its finalized worker report draft.

Parameter	Description
--manifest PATH	[required]
--store PATH	[default: .vllm-sr/evaluation-store]
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark suite-install

Usage: vllm-sr benchmark suite-install [OPTIONS]

Install a normalized suite as exploratory E0 evidence.

Parameter	Description
--request FILE	[required]
--bundle DIRECTORY	[required]
--source-root DIRECTORY	Ignored directory containing the exact-pinned benchmark checkout(s). [required]
--export-root DIRECTORY	Frozen export re-parsed to verify deterministic normalization; this does not attest upstream benchmark execution.
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark suite-list

Usage: vllm-sr benchmark suite-list [OPTIONS]

List browser-safe suite metadata without private artifact references.

Parameter	Description
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark suite-normalize

Usage: vllm-sr benchmark suite-normalize [OPTIONS]

Parse one native export into exploratory E0 replay evidence.

Parameter	Description
--adapter TEXT	[required]
--suite-id TEXT	[required]
--suite-name TEXT	—
--source-root DIRECTORY	Ignored directory containing the exact-pinned benchmark checkout(s). [required]
--export-root DIRECTORY	Directory containing the adapter's frozen native export shape. Its origin is not attested by this command. [required]
--output DIRECTORY	New immutable directory for request.json and the normalized bundle. [required]
--help	Show this message and exit. Default: false.

vllm-sr benchmark suite-show

Usage: vllm-sr benchmark suite-show [OPTIONS] SUITE_ID

Print the immutable operator manifest for one installed suite.

Parameter	Description
SUITE_ID	Required argument. Type: text.
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark validate

Usage: vllm-sr benchmark validate [OPTIONS]

Validate a fixed evaluation manifest without executing it.

Parameter	Description
--manifest PATH	[required]
--suite-store DIRECTORY	[default: .vllm-sr/evaluation-suites]
--help	Show this message and exit. Default: false.

vllm-sr benchmark verify-source

Usage: vllm-sr benchmark verify-source [OPTIONS]

Verify an ignored external source checkout against its exact pin.

Parameter	Description
--adapter TEXT	[required]
--source-root DIRECTORY	[required]
--help	Show this message and exit. Default: false.

vllm-sr completion

Usage: vllm-sr completion [OPTIONS] [COMMAND] [ARGS]...

Generate or install shell completion for vllm-sr.

Examples:
    vllm-sr completion show bash         # Print bash completion script
    vllm-sr completion show zsh          # Print zsh completion script
    vllm-sr completion install           # Auto-install for current shell
    vllm-sr completion install bash      # Install bash completions

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr completion install

Usage: vllm-sr completion install [OPTIONS] [bash|zsh|fish]

Install shell completions into your shell configuration.

Automatically appends the completion setup to your shell's rc file (~/.bashrc, ~/.zshrc) or writes to the fish completions directory. Safe to run multiple times — skips if already configured.

Examples:
    vllm-sr completion install           # Auto-detect shell
    vllm-sr completion install bash      # Install for bash
    vllm-sr completion install zsh       # Install for zsh
    vllm-sr completion install fish      # Install for fish

Parameter	Description
[SHELL]	Optional argument. Type: choice. Choices: bash, zsh, fish.
--help	Show this message and exit. Default: false.

vllm-sr completion show

Usage: vllm-sr completion show [OPTIONS] [bash|zsh|fish]

Print the completion script for a shell.

Outputs the completion script to stdout. If SHELL is omitted, the command attempts to detect the current shell automatically.

Examples:
    vllm-sr completion show bash
    vllm-sr completion show zsh
    vllm-sr completion show fish
    eval "$(vllm-sr completion show zsh)"

Parameter	Description
[SHELL]	Optional argument. Type: choice. Choices: bash, zsh, fish.
--help	Show this message and exit. Default: false.

vllm-sr config

Usage: vllm-sr config [OPTIONS] [COMMAND] [ARGS]...

Print generated configuration or run config subcommands.

Examples:

vllm-sr config envoy
vllm-sr config router
vllm-sr config init --output config.yaml
vllm-sr config envoy --config my-config.yaml
vllm-sr config migrate --config old.yaml
vllm-sr config import --from openclaw --source openclaw.json

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr config apply

Usage: vllm-sr config apply [OPTIONS]

Plan, compare-and-swap, persist, and hot-reload a configuration.

Parameter	Description
--config FILE	[default: config.yaml]
--mode CHOICE	[default: replace] Choices: replace, merge.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	Environment variable containing the Router management bearer token. [default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr config envoy

Usage: vllm-sr config envoy [OPTIONS]

Print the generated Envoy configuration.

Parameter	Description
--config TEXT	Path to config file (default: config.yaml) Default: config.yaml.
--help	Show this message and exit. Default: false.

vllm-sr config get

Usage: vllm-sr config get [OPTIONS]

Read the active canonical configuration from a Router.

Parameter	Description
--format CHOICE	[default: yaml] Choices: json, yaml.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	Environment variable containing the Router management bearer token. [default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr config import

Usage: vllm-sr config import [OPTIONS]

Import a supported external config source into canonical v0.3 YAML.

Parameter	Description
--from CHOICE	Import source type. [required] Choices: openclaw.
--source TEXT	Path to the source config file. Defaults to OpenClaw discovery order.
--target TEXT	Path to the target canonical config file. [default: config.yaml]
--force	Overwrite existing backup files for the source or target paths. Default: false.
--help	Show this message and exit. Default: false.

vllm-sr config init

Usage: vllm-sr config init [OPTIONS]

Create a minimal canonical configuration template.

Parameter	Description
--output TEXT	Path for the new canonical configuration template. [default: config.yaml]
--force	Overwrite the output file if it already exists. Default: false.
--help	Show this message and exit. Default: false.

vllm-sr config migrate

Usage: vllm-sr config migrate [OPTIONS]

Migrate a legacy or mixed config file to canonical v0.3 YAML.

Parameter	Description
--config TEXT	Path to source config file (default: config.yaml) Default: config.yaml.
--output TEXT	Path for migrated canonical config (default: <config>.migrated.yaml)
--force	Overwrite the output file if it already exists. Default: false.
--help	Show this message and exit. Default: false.

vllm-sr config plan

Usage: vllm-sr config plan [OPTIONS]

Validate and plan an exact remote mutation without changing the Router.

Parameter	Description
--config FILE	[default: config.yaml]
--mode CHOICE	[default: replace] Choices: replace, merge.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	Environment variable containing the Router management bearer token. [default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr config rollback

Usage: vllm-sr config rollback [OPTIONS] VERSION

Compare-and-swap the active configuration to a backup version.

Parameter	Description
VERSION	Required argument. Type: text.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	Environment variable containing the Router management bearer token. [default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr config router

Usage: vllm-sr config router [OPTIONS]

Print the canonical router configuration.

Parameter	Description
--config TEXT	Path to config file (default: config.yaml) Default: config.yaml.
--help	Show this message and exit. Default: false.

vllm-sr config schema

Usage: vllm-sr config schema [OPTIONS]

Discover the canonical config contract progressively.

Parameter	Description
--endpoint TEXT	Read the contract from a running Router management origin instead of the schema bundled with this CLI.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--full	Print the complete JSON Schema instead of the compact index. Default: false.
--section PATH	Print one config path and only its referenced definitions.
--surface KIND:NAME	Print one signal, algorithm, plugin, or projection contract.
--expanded	Include the selected section's self-contained JSON Schema. Default: false.
--help	Show this message and exit. Default: false.

vllm-sr config validate

Usage: vllm-sr config validate [OPTIONS]

Validate configuration file.

Examples:

vllm-sr config validate
vllm-sr config validate --config my-config.yaml

Parameter	Description
--config TEXT	Path to config file (default: config.yaml) Default: config.yaml.
--endpoint TEXT	Also validate with this running Router's authoritative parser.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr config versions

Usage: vllm-sr config versions [OPTIONS]

List immutable configuration backup versions.

Parameter	Description
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	Environment variable containing the Router management bearer token. [default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr dashboard

Usage: vllm-sr dashboard [OPTIONS]

Open the dashboard in your default web browser.

Examples:

vllm-sr dashboard                   # Docker dashboard
vllm-sr dashboard --target k8s      # Show K8s address and port forward
vllm-sr dashboard --no-open

Parameter	Description
--no-open	Don't open browser, just show URL Default: false.
--target TEXT	Deployment target: docker, k8s (default: docker)
--namespace TEXT	Kubernetes namespace (k8s target only)
--context TEXT	kubectl / Helm context (k8s target only)
--runtime CHOICE	Container runtime for the local Docker target: docker, podman. Equivalent to setting CONTAINER_RUNTIME=<runtime>. Has no effect on the k8s target. Choices: docker, podman.
--help	Show this message and exit. Default: false.

vllm-sr logs

Usage: vllm-sr logs [OPTIONS] {envoy|router|dashboard}

Show logs from vLLM Semantic Router service.

Examples:

vllm-sr logs envoy
vllm-sr logs router
vllm-sr logs dashboard
vllm-sr logs envoy --follow
vllm-sr logs router -f
vllm-sr logs router --target k8s        # Kubernetes logs
vllm-sr logs router --target k8s -f     # Follow K8s logs

Parameter	Description
SERVICE	Required argument. Type: choice. Choices: envoy, router, dashboard.
-f, --follow	Follow log output Default: false.
--target TEXT	Deployment target: docker, k8s (default: docker)
--namespace TEXT	Kubernetes namespace (k8s target only)
--context TEXT	kubectl / Helm context (k8s target only)
--runtime CHOICE	Container runtime for the local Docker target: docker, podman. Equivalent to setting CONTAINER_RUNTIME=<runtime>. Has no effect on the k8s target. Choices: docker, podman.
--help	Show this message and exit. Default: false.

vllm-sr optimize

Usage: vllm-sr optimize [OPTIONS] COMMAND [ARGS]...

Analyze routing evidence and produce candidate recipe changes.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr optimize recipe-learning

Usage: vllm-sr optimize recipe-learning [OPTIONS]

Analyze replay and outcomes to produce recipe-learning artifacts.

Parameter	Description
--replay-file FILE	Router replay JSON file. Accepts a router_replay.list payload or a record array.
--endpoint TEXT	Router management base URL (origin or /api/v1). Defaults to http://localhost:8080 when --replay-file is omitted. Uses VSR_MGMT_TOKEN for bearer auth when set.
--cases-file FILE	Optional eval cases JSON with replay_id/request_id plus expected_decision or expected_model.
--recipe-file FILE	Optional current recipe YAML used to materialize complete candidate recipe variants.
--limit INTEGER	Replay records to fetch from the endpoint. [default: 100]
--output-dir DIRECTORY	Directory for metrics/findings/patch/seed-pack artifacts.
--json	Print the full recipe-learning artifact. Default: false.
--report-only	Compute metrics and findings without generating recipe patches. Default: false.
--timeout INTEGER	HTTP request timeout in seconds. [default: 15]
--help	Show this message and exit. Default: false.

vllm-sr recipe

Usage: vllm-sr recipe [OPTIONS] COMMAND [ARGS]...

Validate, plan, apply, inspect, or package routing recipes.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr recipe apply

Usage: vllm-sr recipe apply [OPTIONS] RECIPE_FILE

Validate then compare-and-swap one recipe into the active config.

Parameter	Description
RECIPE_FILE	Required argument. Type: file.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr recipe builtin

Usage: vllm-sr recipe builtin [OPTIONS] COMMAND [ARGS]...

Discover, export, or bind installed Recipes without a running Router.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr recipe builtin export

Usage: vllm-sr recipe builtin export [OPTIONS] BUNDLE

Export an exact five-file BUNDLE to a new directory.

Parameter	Description
BUNDLE	Required argument. Type: text.
--output-dir DIRECTORY	[required]
--catalog-version TEXT	[default: latest]
--help	Show this message and exit. Default: false.

vllm-sr recipe builtin init

Usage: vllm-sr recipe builtin init [OPTIONS] NAME

Bind one installed recipe to explicit models and validate the new config.

No model, endpoint, capability, or candidate pool is invented. Every recipe decision must meet its declared minimum_candidates before this succeeds.

Parameter	Description
NAME	Required argument. Type: text.
--bundle TEXT	Bundle from recipe builtin list. [required]
--config FILE	Existing canonical provider config to preserve. [required]
--bindings FILE	YAML mapping each decision that calls a backend to modelRefs; omit immediate responses. [required]
--model-name TEXT	Explicit public entrypoint name. [required]
--exclude-decision TEXT	Explicitly omit a named lane for an unavailable capability; creates a recipe derivative. Repeatable. May be repeated.
--output FILE	New config path; existing files are never replaced. Keep beside --config when it references local KB assets. [required]
--catalog-version TEXT	[default: latest]
--help	Show this message and exit. Default: false.

vllm-sr recipe builtin list

Usage: vllm-sr recipe builtin list [OPTIONS]

List installed bundles, recipe names, and required candidate pools as JSON.

Parameter	Description
--catalog-version TEXT	[default: latest]
--help	Show this message and exit. Default: false.

vllm-sr recipe delete

Usage: vllm-sr recipe delete [OPTIONS] NAME

Compare-and-swap deletion of an unreferenced named recipe.

Parameter	Description
NAME	Required argument. Type: text.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr recipe get

Usage: vllm-sr recipe get [OPTIONS] NAME

Read one managed recipe.

Parameter	Description
NAME	Required argument. Type: text.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr recipe list

Usage: vllm-sr recipe list [OPTIONS]

List recipes and the collection ETag.

Parameter	Description
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr recipe pack

Usage: vllm-sr recipe pack [OPTIONS] RECIPE_DIR

Create a deterministic ZIP from an exact five-file RECIPE_DIR.

Parameter	Description
RECIPE_DIR	Required argument. Type: directory.
--output PATH	Archive path or output directory (default: the Recipe parent directory).
--help	Show this message and exit. Default: false.

vllm-sr recipe plan

Usage: vllm-sr recipe plan [OPTIONS] RECIPE_FILE

Validate a recipe and bind the plan to the current config ETag.

Parameter	Description
RECIPE_FILE	Required argument. Type: file.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr recipe validate

Usage: vllm-sr recipe validate [OPTIONS] RECIPE_FILE

Validate a recipe against the running Router without changing config.

Parameter	Description
RECIPE_FILE	Required argument. Type: file.
--endpoint TEXT	Router management base URL; defaults to the local Router API port.
--timeout FLOAT	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.

vllm-sr request

Usage: vllm-sr request [OPTIONS] COMMAND [ARGS]...

Send requests through an Envoy listener.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr request chat

Usage: vllm-sr request chat [OPTIONS] [MESSAGE]...

Send a one-shot chat completion through the Envoy-routed HTTP API.

Uses the first listener port in config.yaml plus the stack port offset. The default model is the namespaced automatic-routing alias vllm-sr/auto.

Examples:

vllm-sr request chat "hello"

vllm-sr request chat --model vllm-sr/auto --prompt "Explain mixture of models"

vllm-sr request chat --json "hello"

Parameter	Description
[MESSAGE]...	Optional argument. Type: text. Accepts multiple values.
--prompt TEXT	User message (alternative to a positional prompt).
--model TEXT	Model name sent to the router. [default: vllm-sr/auto]
--system TEXT	Optional system message prepended to the conversation.
--config TEXT	Config file used to resolve listener host port (Docker default only). [default: config.yaml]
--base-url TEXT	Explicit routed listener origin or OpenAI /v1 base URL for a remote or port-forwarded stack.
--json	Print the raw JSON response instead of assistant text. Default: false.
--timeout FLOAT	HTTP timeout in seconds for the completion request. [default: 120.0]
--temperature FLOAT	Optional sampling temperature passed through to the API.
--target TEXT	Deployment target (docker or k8s).
--help	Show this message and exit. Default: false.

vllm-sr route

Usage: vllm-sr route [OPTIONS] COMMAND [ARGS]...

Preview routing decisions or probe the routed inference path.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr route preview

Usage: vllm-sr route preview [OPTIONS]

Preview signals and the selected route without calling a model backend.

Parameter	Description
--prompt TEXT	Plain text prompt to evaluate.
--messages TEXT	OpenAI-style messages JSON array string.
--model TEXT	Routing model or entrypoint whose recipe should be evaluated.
--endpoint TEXT	Router management origin or /api/v1 root; defaults to the local management port.
--token-env TEXT	Environment variable containing the Router management bearer token. [default: VSR_MGMT_TOKEN]
--trace / --no-trace	Include per-decision routing trace trees. Default: false.
--json	Print the full JSON response payload. Default: false.
--timeout FLOAT	HTTP request timeout in seconds. [default: 15.0]
--help	Show this message and exit. Default: false.

vllm-sr route probe

Usage: vllm-sr route probe [OPTIONS]

Probe a real route and assert complete assistant delivery for expected 2xx.

Parameter	Description
--prompt TEXT	Plain-text user prompt.
--messages TEXT	OpenAI-style messages JSON array string.
--model TEXT	[default: vllm-sr/auto]
--config TEXT	[default: config.yaml]
--base-url TEXT	Explicit Envoy listener origin or OpenAI /v1 base URL; otherwise derive it from --config.
--api-key-env TEXT	Environment variable containing the bearer token; omitted when unset. [default: OPENAI_API_KEY]
--temperature FLOAT	—
--max-completion-tokens INTEGER RANGE	Completion token budget, including reasoning; omitted uses the backend default. [x>=1]
--timeout FLOAT	[default: 120.0]
--target TEXT	Deployment target used for URL resolution.
--debug / --no-debug	[default: debug]
--expect-status INTEGER	[default: OK]
--expect-recipe TEXT	—
--expect-decision TEXT	—
--expect-algorithm TEXT	—
--expect-selected-model TEXT	Assert the Router's x-vsr-selected-model receipt header.
--expect-response-model TEXT	Assert the upstream OpenAI response body's top-level model field.
--help	Show this message and exit. Default: false.

vllm-sr serve

Usage: vllm-sr serve [OPTIONS]

Start vLLM Semantic Router.

Serve uses --config or config.yaml and preserves the Dashboard-first setup flow. Connect physical models and publish Mixture-of-Model entrypoints in the Dashboard, then keep the same stack running with this single command.

Virtual models are routing policies. Semantic Router starts Router, Envoy, the Dashboard, and supporting services; it does not download or launch the physical LLM engines referenced by provider backends. Connect user-owned single or multiple model endpoints through one canonical config or the Dashboard.

Ports are configured in the selected config under the listeners section.

Local startup waits up to 1800 seconds for Router readiness, or Dashboard readiness during first-run setup. Use --startup-timeout SECONDS for a different positive budget when model loading or GPU compilation needs more time. The wait begins after containers start. It does not change inference deadlines. Timeout exits the CLI with an error and leaves containers available for inspection.

DEPLOYMENT TARGETS:

docker  - Local Docker deployment (default)
k8s     - Kubernetes deployment via Helm

MODEL SELECTION ALGORITHMS:

static     - Use first configured model (default, no learning)
router_dc  - Query-model matching via embedding similarity
automix    - Cost-quality optimization using POMDP
hybrid     - Combine multiple methods with configurable weights
workflows  - Router Flow static/dynamic micro-agent orchestration
latency_aware - TPOT/TTFT percentile-aware selection
knn        - KNN selector using shared ML model-selection settings
kmeans     - KMeans selector using shared ML model-selection settings
svm        - SVM selector using shared ML model-selection settings
mlp        - MLP selector using shared ML model-selection settings
multi_factor - Quality, latency, cost, and load scoring

Cross-request learning lives under global.router.learning.adaptation and global.router.learning.protection instead of --algorithm.

Examples:

# Dashboard-first setup or an existing ./config.yaml
vllm-sr serve
# User-owned single or multi-model topology
vllm-sr serve --config my-models.yaml
# Explicitly replace Dashboard-edited runtime state from reviewed source YAML
vllm-sr serve --config my-models.yaml --replace-active-config
# Deploy a user-owned config to Kubernetes
vllm-sr serve --target k8s --config my-models.yaml --namespace my-ns
# Runtime policy and image overrides
vllm-sr serve --algorithm latency_aware
vllm-sr serve --image-pull-policy always
vllm-sr serve --readonly
vllm-sr serve --minimal
vllm-sr serve --log-level debug
# AMD ROCm image, device passthrough, and router internal GPU defaults
vllm-sr serve --platform amd
vllm-sr serve --platform amd --startup-timeout 7200
VLLM_SR_AMD_ROUTER_VISIBLE_DEVICES=7 vllm-sr serve --platform amd

Parameter	Description
--config TEXT	Path to the Router configuration. [default: config.yaml]
--replace-active-config	Replace this local Docker stack's active runtime config from --config, discarding Dashboard edits. Default: false.
--image TEXT	Docker image to use (default: ghcr.io/vllm-project/semantic-router/vllm-sr:latest)
--router-image TEXT	Docker image for the router container (Docker target only; defaults to --image or VLLM_SR_IMAGE)
--envoy-image TEXT	Docker image for the Envoy container (Docker target only; defaults to --image or VLLM_SR_IMAGE)
--dashboard-image TEXT	Docker image for the dashboard container (Docker target only; defaults to --image or VLLM_SR_IMAGE)
--image-pull-policy CHOICE	Image pull policy: always, ifnotpresent, never (default: always) Choices: always, ifnotpresent, never. Default: always.
--startup-timeout SECONDS	Local Docker startup readiness budget in seconds, including model loading and compilation (default: 1800). [x>=1]
--readonly	Run dashboard in read-only mode (disable config editing, allow playground only) Default: false.
--minimal	Start in minimal mode: only router + envoy, no dashboard or observability (Jaeger, Prometheus, Grafana) Default: false.
--log-level CHOICE	Router log level override (debug, info, warn, error, dpanic, panic, fatal) Choices: debug, info, warn, warning, error, dpanic, panic, fatal.
--platform TEXT	Platform for local Docker GPU deployments: 'amd' enables ROCm passthrough, 'nvidia' enables NVIDIA GPU passthrough (--gpus all). Serve defaults to the matching GPU image (ROCm / CUDA) unless --image or VLLM_SR_IMAGE is provided. Internal models default to GPU, except AMD semantic embeddings retain their configured use_cpu value (default true). MIGraphX mmBERT embeddings require an explicit model binding and deployment with an input token budget. Set VLLM_SR_<PLATFORM>_PRESERVE_CPU=1 to keep CPU settings. For Kubernetes, configure GPU images and resources through a Helm profile or the operator.
--algorithm CHOICE	Request-time base algorithm override for payload-safe algorithms: static, router_dc, automix, hybrid, workflows, latency_aware, knn, kmeans, svm, mlp, multi_factor. Algorithms that require an authored payload remain available in config.yaml. Cross-request learning uses global.router.learning.adaptation/protection. Choices: static, router_dc, automix, hybrid, workflows, latency_aware, knn, kmeans, svm, mlp, multi_factor.
--target TEXT	Deployment target: docker, k8s (default: docker)
--namespace TEXT	Kubernetes namespace (k8s target only)
--context TEXT	kubectl / Helm context (k8s target only)
--profile TEXT	Deployment profile: dev, prod (k8s target only). Selects values-<profile>.yaml defaults.
--chart-dir TEXT	Path to Helm chart directory (k8s target only)
--runtime CHOICE	Container runtime for the local Docker target: docker, podman. Equivalent to setting CONTAINER_RUNTIME=<runtime>. Has no effect on the k8s target. Choices: docker, podman.
--recipe-env NAME	Explicitly bind one host environment variable for the active Recipe. Repeat for multiple names; NAME=value is rejected. May be repeated.
--help	Show this message and exit. Default: false.

vllm-sr status

Usage: vllm-sr status [OPTIONS] [envoy|router|dashboard|all]

Show status of vLLM Semantic Router services.

Examples:

vllm-sr status              # Show all services (Docker)
vllm-sr status all          # Show all services
vllm-sr status router       # Show router status
vllm-sr status dashboard    # Show dashboard status
vllm-sr status --target k8s # Show Kubernetes status

Parameter	Description
[SERVICE]	Optional argument. Type: choice. Choices: envoy, router, dashboard, all. Default: all.
--target TEXT	Deployment target: docker, k8s (default: docker)
--namespace TEXT	Kubernetes namespace (k8s target only)
--context TEXT	kubectl / Helm context (k8s target only)
--runtime CHOICE	Container runtime for the local Docker target: docker, podman. Equivalent to setting CONTAINER_RUNTIME=<runtime>. Has no effect on the k8s target. Choices: docker, podman.
--help	Show this message and exit. Default: false.

vllm-sr stop

Usage: vllm-sr stop [OPTIONS]

Stop vLLM Semantic Router.

Examples:

vllm-sr stop                # Stop Docker stack
vllm-sr stop --target k8s   # Uninstall Helm release

Parameter	Description
--target TEXT	Deployment target: docker, k8s (default: docker)
--namespace TEXT	Kubernetes namespace (k8s target only)
--context TEXT	kubectl / Helm context (k8s target only)
--runtime CHOICE	Container runtime for the local Docker target: docker, podman. Equivalent to setting CONTAINER_RUNTIME=<runtime>. Has no effect on the k8s target. Choices: docker, podman.
--help	Show this message and exit. Default: false.

vllm-sr storage

Usage: vllm-sr storage [OPTIONS] COMMAND [ARGS]...

Inspect Router storage or manage local storage credentials.

Parameter	Description
--help	Show this message and exit. Default: false.

vllm-sr storage rotate

Usage: vllm-sr storage rotate [OPTIONS]

Replace this stack's storage credentials with freshly generated ones.

Rotation generates new values, applies them to Postgres with ALTER ROLE and to Redis by rebuilding it against the same named volume, then asks you to re-run serve so Router picks the new values up.

Router has to be restarted, and this command deliberately does not do it for you. Router receives the credentials as environment values captured when its container was created, so restart would bring back the old ones and only a re-create picks up the new ones -- and re-creating it here would have to guess the images, profile, and Recipe bindings you originally served with. Re-run your own serve command instead.

Until you do, the stack is degraded: ALTER ROLE takes effect at once, so connections Router already holds keep working while every new one fails. The order is forced -- restarting Router first would start it on a credential Postgres has not accepted yet -- so run the two steps back to back and plan the rotation for a moment when a brief restart is acceptable.

The scope is one stack, resolved from VLLM_SR_STACK_NAME exactly like serve, stop, and status. Rotate other stacks one at a time. There is deliberately no cross-stack mode: a failure partway through would leave some stacks revoked and others not, with no value left to roll back to.

Examples:

vllm-sr storage rotate
VLLM_SR_STACK_NAME=staging vllm-sr storage rotate

Parameter	Description
--config TEXT	Config file whose directory holds the runtime state (default: config.yaml, matching vllm-sr serve).
--runtime CHOICE	Container runtime for the local Docker target: docker, podman Choices: docker, podman.
--help	Show this message and exit. Default: false.

vllm-sr storage vector-stores

Usage: vllm-sr storage vector-stores [OPTIONS]

List vector stores known to the Router management API.

Parameter	Description
--endpoint TEXT	Router management origin or /api/v1 root.
--timeout INTEGER	[default: 15]
--token-env TEXT	[default: VSR_MGMT_TOKEN]
--help	Show this message and exit. Default: false.