Decision 3.0 on Apple Silicon: Text, Image and Video Decisions on a Mac
d3-lite sorts 1,000 sample pull requests into the vLLM-SR workgroups, on an M5 Pro. Right: GPU load and the per-PR log.
Decision 3.0 runs on a Mac.
Decision 3.0 brought text, image and video decisions to one open model family. We converted the three smaller sizes (d3-lite 0.8B, d3-nano 2B and d3-mini 4B) to Core ML and built demos for text, screenshots and live video.
A decision made on the Mac has no per-call cost and no network round trip, and the pull request, web page or video never leaves the machine. That fits alongside vLLM-SR: routine decisions can be answered on the device, and the ones that need a bigger model can go to the router.
Get the Core ML models → · Swift runtime → · Watch the demo on X →
How the conversion works
Each d3 model is a Qwen3.5 backbone with a small FP32 readout that scores every option at the last prompt token, one forward pass per question. Core ML runs the heavy part, the backbone and the vision tower. The host app runs the light steps around it: a table lookup for the embeddings, arithmetic for the positions, and one small matrix multiply for the readout.
In detail:
- Text graph on Core ML. The backbone takes token embeddings as input, so image and video tokens can be spliced in before it runs. Prompts are right-padded to a few fixed lengths. That is exact for d3: the Gated DeltaNet layers are causal and never see the padding, and the full-attention layers mask it out.
- Vision tower on Core ML. It runs in fp32 with fixed patch counts and masked padding. For video, a block-diagonal mask keeps attention inside each frame pair, as upstream does.
- Host side. Token embeddings, M-RoPE positions and the 255-way readout run in numpy (or Swift) in FP32. Prompts come from upstream's own request preparation, run on a d3 with no weights loaded, so every prompt, timestamp and position matches upstream exactly.
- 8-bit weights. At fp16, d3-mini's text graph is 6.7 GB per length bucket. Two buckets loaded at once made the GPU page on a 24 GB Mac, at 8–12 s per step. With 8-bit weights both stay resident.
Conversion uses PyTorch 2.7 and coremltools 9.0. We checked the Core ML release against upstream PyTorch (bf16 on the Mac's GPU) on two public benchmarks: the JevBench public suite, run through its unchanged System One adapter and scorer as in the System One Auto post, and the typed-decisions TEST split, scored against its gold labels.
| d3-lite | d3-nano | d3-mini | |
|---|---|---|---|
| JevBench public (231 tasks), PyTorch → Core ML | 73.2% → 74.0% | 75.3% → 75.8% | 81.8% → 81.8% |
| typed-decisions TEST (2,000 questions), PyTorch → Core ML | 51.6% → 51.6% | 62.7% → 62.7% | 73.6% → 73.9% |
| Median JevBench request, PyTorch → Core ML | 204 → 53 ms | 173 → 69 ms | 438 → 169 ms |
| Download | 3.7 GB | 10.5 GB | 19.6 GB |
Core ML scores the same as upstream, and each request is 2.5–4× faster. On a Mac, upstream PyTorch falls back to Transformers' reference code for the DeltaNet layers, since the fast kernels it uses on NVIDIA GPUs aren't available.
All numbers are from one MacBook Pro (M5 Pro, 24 GB), running on the GPU. d3-nano is the middle option: close to d3-lite's speed, and 11 points above it on typed-decisions.
Method and limits
- Both backends ran one request at a time. Request times include a local HTTP round trip.
- JevBench accuracy is exact-label accuracy on the public suite, not the official sealed leaderboard or its composite score. 37 of its tasks are longer than 1,536 tokens (the longest is 3,970), so the release includes a 4,096-token text graph.
- typed-decisions gold labels come from several annotators who often disagree, so accuracy there sits well below JevBench for every model.
Demos
d3-lite handles the demo where speed matters; d3-mini handles screenshots and video.
Triage pull requests (d3-lite, on X). The video at the top. For each of 1,000 sample pull requests, d3-lite reads the title, the start of the description and the changed files, then answers two questions: which of the seven vLLM-SR workgroups owns it, and what kind of change it is. Each PR takes 0.12 s, so 8.3 PRs and 16.5 decisions per second. On real vLLM-SR pull requests, d3-mini matched the maintainers' workgroup label on 134 of 168 and the change-type tag on 153 of 182.
Goal: open a book in the Travel category with a five-star rating. d3-mini picks each click from a screenshot.
Drive a real website (d3-mini). d3-mini gets a goal and a screenshot of books.toscrape.com with numbered marks on the links. It picks the next click, reads the books on the page, and stops when one meets the goal. It reached all 17 goals in a 17-goal test, at about 2.2 s per click. d3 reads values well but compares them less reliably: one pick across a whole page reached 9 of 17. Reading each book card and comparing in code reached all 17.
Each time d3-mini is free, it looks at the last 4 seconds and the timeline grows from its answers. Video: diagonal crosswalk at Yonge & Dundas, Toronto (Raysonho, Wikimedia Commons, CC0).
Watch live video (d3-mini). As the clip plays, d3-mini looks at the last 4 seconds (8 frames at its native 2 fps) and answers whether the cars are moving and whether someone is cycling. Each look takes 1.1 s on Core ML, against 2.4 s for PyTorch on the same Mac, and answers stay about 2 seconds behind the live video.
Try it
The Core ML models for all three sizes are in FluidInference/d3-coreml, and each folder has a Python host built on coremltools. For Swift apps, D3Manager in FluidUse downloads a model and answers requests.
Thanks to the vLLM Semantic Router team for early access to Decision 3.0.