跳到主要内容
版本：最新版（main）

Decision-1.0-Kai-0.6B

One of the three 0.6B Decision 1.0 encoders (Kai, Lex and Route), on the Vela encoder backbone. It runs well on a CPU.

PropertyValue
Repositoryvllm-sr/Decision-1.0-Kai-0.6B, pinned at 79263ba4c4be
FamilyDecision 1.0
Parameters0.572B loaded
BackboneModernBERT
Precision (--dtype auto)FP32 on every device
Weightsabout 2.3 GB at FP32
Device memory4 GiB or more
Readstext

Question types​

TypeAnswers
choiceYes
noulYes
scoreYes
setYes, asked as one noul per label
spanNo: bind span questions, such as PII and hallucination, to a span-capable model

Built-in questions​

The Decision 1.0 family answers the Router's built-in questions zero-shot, from their canonical wording; the support matrix shows what that covers.

备注

This model has no calibrated thresholds: a built-in question without a predicate uses 0.5, and the configuration load warns with uncalibrated_preset.

Hardware​

Each time it loads, the model compares its answers with the reference answers recorded for its revision. They are recorded for:

  • CPU (cpu)
  • NVIDIA Hopper (sm_90) (cuda:sm_90)
  • AMD GPU (ROCm) (rocm)

Where a hardware reference and its device class's are both recorded, the hardware one is checked.

Choose a model lists the validated hardware.

Quantization​

The runtime loads no quantized packages of the Decision 1.0 family: it runs at its published precision. Batches that merge requests run a pre-packed FP32 copy of its linear layers on x86 CPUs with oneDNN, which its records show changes no decision.

Deploy it​

Serve it on its own in Engine mode, to ask it directly:

vllm-sr serve vllm-sr/Decision-1.0-Kai-0.6B --engine --platform cpu

Or make it the Router's decision model:

config.yaml
global:
  model_catalog:
    deployments:
      primary:
        provider: model_runtime
        artifact: vllm-sr/Decision-1.0-Kai-0.6B
        device: auto
      spans:
        provider: model_runtime
        artifact: vllm-sr/vela3-fast
    bindings:
      pii: spans
      hallucination: spans
    system:
      decision_model: {deployment: primary}

device: auto takes the first validated GPU with enough free memory, else the CPU; name rocm:0 or cuda:0 to pin a GPU.

This model answers no span questions, so the example keeps pii and hallucination on a span-capable deployment; without those bindings a pii rule fails to load with span_required.

To run it on another host and attach the Router to it:

vllm-srun serve vllm-sr/Decision-1.0-Kai-0.6B --device auto --host 0.0.0.0 --port 8100

Then set endpoint: http://<host>:8100 and model: Decision-1.0-Kai-0.6B on the deployment; see Attach to a runtime you run.

Learn more​