Skip to main content

One post tagged with "claude-code"

View All Tags

vSR: vLLM Semantic Router Inside Claude Code

· 19 min read
Theo Hsiung
Associate Research Scientist @ Delta Electronics
Abhinav Mahajan
Software Engineer at Barclays

The vLLM Semantic Router mascot leads six bodyguards, one per vSR app, to the Claude Code mascot: a signpost for router, a ladder for escalate, a marker for redact, a shield for shield, a stop sign for guard and a magnifying glass for check.

A coding agent does three things all day: it hands work to subagents, it reads files, and it runs commands. Each one carries a cost that the session never shows you.

Subagents run at your model's price. A general-purpose subagent starts on your session's model. Listing files, counting lines or renaming a variable costs the same as a hard design question, because nothing asks how hard the task is before the subagent starts.

Reads carry more than code. The agent reads what it is pointed at: a support ticket with a customer's email and card number, a config file with an API key, a vendored README with an instruction to AI agents hidden in a comment. Whatever it reads becomes part of the conversation, and what a file says can steer what the agent does next.

Commands blur together. A long session runs dozens of Bash commands, and almost all of them are harmless. A git reset --hard chained with a force push is not, yet in a long run of prompts, or in auto mode with none at all, it looks like the rest.

Each of these is a routing question: which model should take this task, what in this output should the model never see, which command needs a person. vLLM Semantic Router answers questions like these for every request at the gateway. vSR asks them inside the coding agent, at the moment each one happens.

vSR is a Claude Code plugin that puts vLLM Semantic Router's decision model inside the harness. One small model, vela3-lite, answers the questions a router would ask: how hard is this task, is there personal data in this output, is this file trying to instruct the agent, does this command need a person. Your requests still go to Claude as usual.