SemanticRouter CRD Reference
SemanticRouter is the Operator-owned resource for deploying a Router and
binding it to Kubernetes model services.
apiVersion: vllm.ai/v1alpha1
kind: SemanticRouter
metadata:
name: my-router
spec: {}
This page summarizes the top-level contract. The installed CRD is authoritative for nested OpenAPI validation and defaults:
kubectl explain semanticrouter.spec --recursive
The source schema is available in
semanticrouter_types.go
and the generated CRD in
vllm.ai_semanticrouters.yaml.
Top-level spec fields
| Field | Purpose |
|---|---|
image | Router image repository, tag, registry prefix, and pull policy. |
replicas | Fixed replica count when autoscaling is not controlling replicas. |
imagePullSecrets | Registry credentials referenced by name. |
serviceAccount | Create or select the workload ServiceAccount. |
service | Service type and API, gRPC, and metrics ports. |
resources | Container CPU, memory, and other resource requests/limits. |
persistence | Model-storage PVC settings or an existing claim. |
config | Operator configuration adapters and canonical recipes, entrypoints, plugins and models. |
toolsDb | Function-tool records materialized for Router tool selection. |
vllmEndpoints | Model services discovered as canonical providers and model cards. |
autoscaling | HPA enablement and CPU/memory targets. |
startupProbe, livenessProbe, readinessProbe | Workload probe tuning. |
securityContext, podSecurityContext | Container and pod security settings. |
podAnnotations | Additional annotations, including scraper integration. |
nodeSelector, tolerations, affinity | Pod scheduling constraints. |
env, args | Additional Router environment variables and arguments. |
gateway | Reference to an existing Kubernetes Gateway. |
openshift | OpenShift Route behavior. |
ingress | Kubernetes Ingress configuration. |
vllmEndpoints
Each entry creates one logical provider model and one backend reference:
spec:
vllmEndpoints:
- name: qwen-backend
model: qwen/assistant
reasoning:
family: qwen3
backend:
type: service
service:
name: qwen-vllm
namespace: model-serving
port: 8000
weight: 1
Supported backend types are service, kserve, and llamastack. Optional
loras declare adapters exposed by the generated routing model card. The first
resolved model becomes the default unless configuration overrides it.
config
The Operator always renders a canonical v0.5 Router document. Its configuration surface has two parts:
config.recipes,config.entrypoints,config.pluginsandconfig.modelspass through canonical recipes, entrypoints,global.pluginsand the facts of provider models;config.routingis removed and refused;- typed adapter fields such as
response_cache,complexity_rules,reasoning_effort,api,observability, andstreamed_bodyare translated into their canonical provider orgloballocations.
Do not assume an arbitrary local config.yaml key is valid directly under
spec.config. Use the CRD schema and
Configuration Workflows when moving
between local YAML and the Operator.
The deprecated semantic_cache adapter is retained for compatibility;
response_cache is the canonical field. Do not set both.
Model deployments and the decision model
config.model_deployments, config.model_bindings and config.decision_model
become global.model_catalog.deployments, global.model_catalog.bindings and
global.model_catalog.system.decision_model. They take the same forms as a
local configuration; see the model catalog reference.
The decision model must be a model_runtime or systemone deployment, and a
built-in signal that no binding names follows it.
A remote deployment's key never goes in the resource: auth.api_key_env names
a spec.env variable filled from a Secret. The Operator mounts no Secret file
into the Router Pod, so auth.api_key_file is not available.
spec:
env:
- name: SYSTEMONE_KEY
valueFrom:
secretKeyRef: {name: systemone-credentials, key: api-key}
config:
model_deployments:
d3-remote:
provider: systemone
endpoint: https://systemone.example.com/v1/systemone
model: vllm-sr/d3-ultra
schema: vllm-sr/5.0
auth: {api_key_env: SYSTEMONE_KEY}
timeout_ms: 1500
local-vela: {provider: model_runtime, artifact: vllm-sr/vela3-fast, device: cpu}
decision_model: {deployment: d3-remote}
model_bindings:
pii: local-vela
Status
status reports the observed generation, replica counts, conditions, phase,
gateway mode, and detected OpenShift features. Controllers and automation
should use conditions and status.observedGeneration, not only the phase.
kubectl get semanticrouter <name> -o jsonpath='{.status.conditions}'
kubectl get semanticrouter <name> -o jsonpath='{.status.observedGeneration}'