Skip to main content
Version: Latest (main)

SemanticRouter CRD Reference

SemanticRouter is the Operator-owned resource for deploying a Router and binding it to Kubernetes model services.

apiVersion: vllm.ai/v1alpha1
kind: SemanticRouter
metadata:
name: my-router
spec: {}

This page summarizes the top-level contract. The installed CRD is authoritative for nested OpenAPI validation and defaults:

kubectl explain semanticrouter.spec --recursive

The source schema is available in semanticrouter_types.go and the generated CRD in vllm.ai_semanticrouters.yaml.

Top-level spec fields​

FieldPurpose
imageRouter image repository, tag, registry prefix, and pull policy.
replicasFixed replica count when autoscaling is not controlling replicas.
imagePullSecretsRegistry credentials referenced by name.
serviceAccountCreate or select the workload ServiceAccount.
serviceService type and API, gRPC, and metrics ports.
resourcesContainer CPU, memory, and other resource requests/limits.
persistenceModel-storage PVC settings or an existing claim.
configOperator configuration adapters and canonical recipes, entrypoints, plugins and models.
toolsDbFunction-tool records materialized for Router tool selection.
vllmEndpointsModel services discovered as canonical providers and model cards.
autoscalingHPA enablement and CPU/memory targets.
startupProbe, livenessProbe, readinessProbeWorkload probe tuning.
securityContext, podSecurityContextContainer and pod security settings.
podAnnotationsAdditional annotations, including scraper integration.
nodeSelector, tolerations, affinityPod scheduling constraints.
env, argsAdditional Router environment variables and arguments.
gatewayReference to an existing Kubernetes Gateway.
openshiftOpenShift Route behavior.
ingressKubernetes Ingress configuration.

vllmEndpoints​

Each entry creates one logical provider model and one backend reference:

spec:
vllmEndpoints:
- name: qwen-backend
model: qwen/assistant
reasoning:
family: qwen3
backend:
type: service
service:
name: qwen-vllm
namespace: model-serving
port: 8000
weight: 1

Supported backend types are service, kserve, and llamastack. Optional loras declare adapters exposed by the generated routing model card. The first resolved model becomes the default unless configuration overrides it.

config​

The Operator always renders a canonical v0.5 Router document. Its configuration surface has two parts:

  • config.recipes, config.entrypoints, config.plugins and config.models pass through canonical recipes, entrypoints, global.plugins and the facts of provider models; config.routing is removed and refused;
  • typed adapter fields such as response_cache, complexity_rules, reasoning_effort, api, observability, and streamed_body are translated into their canonical provider or global locations.

Do not assume an arbitrary local config.yaml key is valid directly under spec.config. Use the CRD schema and Configuration Workflows when moving between local YAML and the Operator.

The deprecated semantic_cache adapter is retained for compatibility; response_cache is the canonical field. Do not set both.

Model deployments and the decision model​

config.model_deployments, config.model_bindings and config.decision_model become global.model_catalog.deployments, global.model_catalog.bindings and global.model_catalog.system.decision_model. They take the same forms as a local configuration; see the model catalog reference. The decision model must be a model_runtime or systemone deployment, and a built-in signal that no binding names follows it.

A remote deployment's key never goes in the resource: auth.api_key_env names a spec.env variable filled from a Secret. The Operator mounts no Secret file into the Router Pod, so auth.api_key_file is not available.

spec:
env:
- name: SYSTEMONE_KEY
valueFrom:
secretKeyRef: {name: systemone-credentials, key: api-key}
config:
model_deployments:
d3-remote:
provider: systemone
endpoint: https://systemone.example.com/v1/systemone
model: vllm-sr/d3-ultra
schema: vllm-sr/5.0
auth: {api_key_env: SYSTEMONE_KEY}
timeout_ms: 1500
local-vela: {provider: model_runtime, artifact: vllm-sr/vela3-fast, device: cpu}
decision_model: {deployment: d3-remote}
model_bindings:
pii: local-vela

Status​

status reports the observed generation, replica counts, conditions, phase, gateway mode, and detected OpenShift features. Controllers and automation should use conditions and status.observedGeneration, not only the phase.

kubectl get semanticrouter <name> -o jsonpath='{.status.conditions}'
kubectl get semanticrouter <name> -o jsonpath='{.status.observedGeneration}'