d3-mini

A 4B Decision 3.0 model for your own questions about text, images and video, and the base of vela3-mini.

Property	Value
Repository	vllm-sr/d3-mini, pinned at a475367de0d8
Family	Decision 3.0
Parameters	4.54B loaded
Backbone	Qwen3.5 text
Precision (--dtype auto)	BF16 on a GPU, FP32 on a CPU
Weights	about 9.1 GB at BF16, 18.2 GB at FP32
Device memory	16 GiB or more
Reads	text, image, video

Question types

Type	Answers
choice	Yes
noul	Yes
score	Yes
set	Yes, asked as one noul per label
span	No: bind span questions, such as PII and hallucination, to a span-capable model

Rules that read the request's images (input.image) or the current turn's videos (input.video) can resolve to it.

Built-in questions

The Decision 3.0 family answers the Router's built-in questions zero-shot, from their canonical wording; the support matrix shows what that covers.

note

This model has no calibrated thresholds: a built-in question without a predicate uses 0.5, and the configuration load warns with uncalibrated_preset.

Hardware

Each time it loads, the model compares its answers with the reference answers recorded for its revision. They are recorded for:

• CPU (cpu)
• Ascend NPU (npu)
• Ascend 950DT (npu:Ascend950DT)
• AMD GPU (ROCm) (rocm)

Served with --dtype bfloat16, it compares them with the BF16 references recorded for:

• CPU (cpu@bfloat16)
• x86-64 CPU with AVX2 (BF16 emulated) (cpu:x86_64:avx2:bf16_emulated@bfloat16)
• x86-64 CPU with AVX-512 BF16 (cpu:x86_64:avx512:bf16_native@bfloat16)

Where a hardware reference and its device class's are both recorded, the hardware one is checked.

On an NVIDIA GPU no reference is recorded, so the model serves there with its self-check reported as unverified.

Choose a model lists the validated hardware.

Quantization

The Decision 3.0 family loads quantized packages in the compressed-tensors format: FP8 (W8A8), int4 (W4A16, GPTQ or AWQ) and NVFP4 (W4A4). A quantized package is its own model, with its own answers.

Quantization compares the formats and the hardware each needs.

Deploy it

Serve it on its own in Engine mode, to ask it directly:

CPU

vllm-sr serve vllm-sr/d3-mini --engine --platform cpu

Serve it on a CPU from the CPU image, as --platform cpu does: the ROCm image's PyTorch has no CPU LAPACK, which this backbone needs there.

A CPU computes it in FP32: 18.2 GB of weights, twice its BF16 size. On a CPU that computes BF16 natively (AVX-512 BF16 or AMX) a request takes 1.2 to 3.5 times as long as at --dtype bfloat16, the larger sizes the most; on one that emulates BF16 (AVX2) FP32 is 3 to 6 times faster. --dtype bfloat16 is checked against the BF16 reference of its CPU class (see Hardware).

AMD GPU

vllm-sr serve vllm-sr/d3-mini --engine --platform rocm

NVIDIA GPU

vllm-sr serve vllm-sr/d3-mini --engine --platform cuda

Or make it the Router's decision model:

config.yaml

global:
  model_catalog:
    deployments:
      primary:
        provider: model_runtime
        artifact: vllm-sr/d3-mini
        device: auto
      spans:
        provider: model_runtime
        artifact: vllm-sr/vela3-fast
    bindings:
      pii: spans
      hallucination: spans
    system:
      decision_model: {deployment: primary}

device: auto takes the first validated GPU with enough free memory, else the CPU; name rocm:0 or cuda:0 to pin a GPU.

This model answers no span questions, so the example keeps pii and hallucination on a span-capable deployment; without those bindings a pii rule fails to load with span_required.

To run it on another host and attach the Router to it:

vllm-srun serve vllm-sr/d3-mini --device auto --host 0.0.0.0 --port 8100

Then set endpoint: http://<host>:8100 and model: d3-mini on the deployment; see Attach to a runtime you run.

Learn more

• Model card on Hugging Face
• decision3-parity record
• npu-bf16-ascend950dt record
• Decision 3.0: the family's sizes and how they differ