Decision-2.0-Sol-2B

A 2B Decision 2.0 model for questions that need more judgment. Serve it on a GPU, or on a CPU for low traffic.

Property	Value
Repository	vllm-sr/Decision-2.0-Sol-2B, pinned at 64235bef55da
Family	Decision 2.0
Parameters	1.88B loaded
Backbone	Qwen3.5 text
Precision (--dtype auto)	BF16 on a GPU, FP32 on a CPU
Weights	about 3.8 GB at BF16, 7.5 GB at FP32
Device memory	8 GiB or more
Reads	text

Question types

Type	Answers
choice	Yes
noul	Yes
score	Yes
set	Yes, asked as one noul per label
span	No: bind span questions, such as PII and hallucination, to a span-capable model

Built-in questions

The Decision 2.0 family answers the Router's built-in questions zero-shot, from their canonical wording; the support matrix shows what that covers.

备注

This model has no calibrated thresholds: a built-in question without a predicate uses 0.5, and the configuration load warns with uncalibrated_preset.

Hardware

Each time it loads, the model compares its answers with the reference answers recorded for its revision. They are recorded for:

• CPU (cpu)
• NVIDIA Hopper (sm_90) (cuda:sm_90)
• AMD GPU (ROCm) (rocm)

Where a hardware reference and its device class's are both recorded, the hardware one is checked.

Choose a model lists the validated hardware.

Quantization

The runtime loads no quantized packages of the Decision 2.0 family: it runs at its published precision.

Deploy it

Serve it on its own in Engine mode, to ask it directly:

CPU

vllm-sr serve vllm-sr/Decision-2.0-Sol-2B --engine --platform cpu

Serve it on a CPU from the CPU image, as --platform cpu does: the ROCm image's PyTorch has no CPU LAPACK, which this backbone needs there.

AMD GPU

vllm-sr serve vllm-sr/Decision-2.0-Sol-2B --engine --platform rocm

NVIDIA GPU

vllm-sr serve vllm-sr/Decision-2.0-Sol-2B --engine --platform cuda

Or make it the Router's decision model:

config.yaml

global:
  model_catalog:
    deployments:
      primary:
        provider: model_runtime
        artifact: vllm-sr/Decision-2.0-Sol-2B
        device: auto
      spans:
        provider: model_runtime
        artifact: vllm-sr/vela3-fast
    bindings:
      pii: spans
      hallucination: spans
    system:
      decision_model: {deployment: primary}

device: auto takes the first validated GPU with enough free memory, else the CPU; name rocm:0 or cuda:0 to pin a GPU.

This model answers no span questions, so the example keeps pii and hallucination on a span-capable deployment; without those bindings a pii rule fails to load with span_required.

To run it on another host and attach the Router to it:

vllm-srun serve vllm-sr/Decision-2.0-Sol-2B --device auto --host 0.0.0.0 --port 8100

Then set endpoint: http://<host>:8100 and model: Decision-2.0-Sol-2B on the deployment; see Attach to a runtime you run.

Learn more

• Model card on Hugging Face
• rocm-mi325x-parity record
• cuda-decision2-performance record
• Decision 2.0: the family's sizes and how they differ