Skip to main content
Version: Latest (main)

vela3-fast

The default decision model. A CPU-friendly encoder with trained adapters for every built-in routing question, calibrated thresholds, and spans for PII and hallucination checks.

PropertyValue
Repositoryvllm-sr/vela3-fast, pinned at 7cbefd4b92b5
FamilyVela 3.0
Parameters1.52B loaded
BackboneUltraBERT
Precision (--dtype auto)FP32 on every device
Weightsabout 6.1 GB at FP32
Device memory12 GiB or more
Readstext

Question types​

TypeAnswers
choiceYes
noulYes
scoreNo
setYes, asked as one noul per label
spanYes, on the router span head

Built-in questions​

The Vela 3.0 family was trained on the Router's built-in questions, so the Router asks each one in its trained form; the support matrix lists which.

A rule without a predicate takes this model's calibrated threshold:

PresetThreshold
jailbreak (jailbreak)0.88
domain (domain)0.41
pii (pii)0.18
fact check (fact_check)0.9999
user feedback (feedback)0.47
safety (harm)0.52
modality (modality)0.63

Hardware​

Each time it loads, the model compares its answers with the reference answers recorded for its revision. They are recorded for:

  • CPU (cpu)
  • x86-64 CPU with AVX2 (BF16 emulated) (cpu:x86_64:avx2:bf16_emulated)

Where a hardware reference and its device class's are both recorded, the hardware one is checked.

On an AMD GPU or an NVIDIA GPU no reference is recorded, so the model serves there with its self-check reported as unverified.

Choose a model lists the validated hardware.

Quantization​

The runtime loads no quantized packages of the Vela 3.0 family: it runs at its published precision.

Deploy it​

Serve it on its own in Engine mode, to ask it directly:

vllm-sr serve vllm-sr/vela3-fast --engine --platform cpu

Or make it the Router's decision model:

config.yaml
global:
  model_catalog:
    deployments:
      primary:
        provider: model_runtime
        artifact: vllm-sr/vela3-fast
        device: auto
    system:
      decision_model: {deployment: primary}

device: auto takes the first validated GPU with enough free memory, else the CPU; name rocm:0 or cuda:0 to pin a GPU.

To run it on another host and attach the Router to it:

vllm-srun serve vllm-sr/vela3-fast --device auto --host 0.0.0.0 --port 8100

Then set endpoint: http://<host>:8100 and model: vela3-fast on the deployment; see Attach to a runtime you run.

Learn more​