Quickstart
Install vLLM Semantic Router, start the local stack, and send one request.
Requirements
- Linux, macOS, or WSL2
- Python 3.10 or newer
- Docker; Linux can fall back to Podman
For the curl installer, pass --runtime podman to force Podman or
--runtime skip to skip container-runtime preparation. For example:
curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel stable --runtime skip
These are installer options. vllm-sr serve --runtime selects a container
runtime (docker or podman); skip is not a serve runtime.
Install
- curl
- pip
- uv
- Agent
curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade --pre vllm-sr
uv tool install --upgrade --prerelease allow vllm-sr
Copy this prompt into your coding agent:
Install the development channel of vLLM Semantic Router, then configure and verify it on this machine by following the official skill: https://vllm-sr.ai/install/agent/vllm-sr/SKILL.md
The prompt points to the public, self-contained vLLM SR agent skill. See Install with an agent for the workflow and safety boundaries.
Verify the CLI:
vllm-sr --version
The curl installer starts the stack automatically. After a pip or uv install, start it with:
vllm-sr serve
Open http://localhost:8700, add a model endpoint, and activate the generated configuration. Agents can do the same work through the CLI and Router management API without using the Dashboard.
Send a request
curl http://localhost:8899/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "vllm-sr/auto",
"messages": [{"role": "user", "content": "Hello!"}]
}'