Spaces:

---

XciD

/

anthropic-messages-proxy

Like 0

Running on CPU Upgrade

Problem

/v1/messages silently drops model reasoning. Providers on the router return chain-of-thought as either reasoning_content (deepinfra, fireworks-ai, novita, baseten, zai-org, nscale) or reasoning (together, cerebras, groq, ovhcloud, scaleway), but StreamConverter::process_chunk only handles delta.content and delta.tool_calls. Claude Code therefore receives text and tool_use blocks and never a thinking block. The reasoning tokens are still counted in output_tokens, so they are billed and then discarded.

The data already reaches the proxy: handlers/messages.rs reads both fields, but only to build a tracing::debug! line. And convert_responses.rs has a working handle_reasoning() for the OpenAI Responses surface — the Anthropic surface never got the equivalent.

Fix (response path)

• process_chunk: map reasoning / reasoning_content onto a thinking block plus thinking_delta. Placed ahead of the text branch, because Anthropic requires the thinking block to precede the text it belongs to.
• Accept both field names. The name varies by provider and by model — together sends reasoning for GLM-5.2 and MiniMax-M3, but reasoning_content for Kimi-K3 — so keying off the provider alone would silently drop reasoning on some combinations.
• close_current_block: emit signature_delta before closing a thinking block. Open models have nothing to sign, so the value is a constant marker; clients treat the signature as opaque, and some expect the field to be present.
• aggregate_message: add ThinkingDelta / SignatureDelta arms. Without them the non-streaming path (stream: false, used by the auto-mode classifier) would keep dropping reasoning even after streaming was fixed.

No new types were needed. MessageContentType::Thinking, ThinkingDelta, SignatureDelta, MessageContentDelta.thinking / .signature and ChunkDelta.reasoning / .reasoning_content all already existed — this is wiring, not new plumbing.

Deliberately unchanged (request path)

src.thinking is still not forwarded, and inbound thinking blocks are still dropped. The obvious mapping is OpenRouter's reasoning field, but it is not portable across the router's backends:

• fireworks-ai rejects the request with 400 "Extra inputs are not permitted, field: 'reasoning'" — both as a request parameter and as reasoning on an assistant message.
• novita accepts it and ignores it: reasoning_content still streams under reasoning: {enabled: false}.

Forwarding it would cost a provider and buy no control. This matters more after this change than before it: clients replay thinking blocks on the following turn, so dropping them in convert_request.rs is what keeps multi-turn working on every backend.

The two stale not supported by HuggingFace router comments are replaced with this reasoning, and two tests assert reasoning is never sent upstream. Otherwise those comments read as TODOs and the next change to them breaks fireworks-ai.

Testing

19 unit tests (4 new, built on shared run() / starts() / delta_types() helpers), covering both field names, block ordering and indices, the signature, thinking alongside tool_use, and the request-path invariant.

Coverage was checked by mutation rather than by line count — each of these breaks at least one test: an always-true signature guard, dropping either half of the reasoning / reasoning_content fallback, removing the ThinkingDelta arm from aggregate_message, a no-op reasoning branch, and a block_index that never increments.

Verified against the live router on novita, fireworks-ai, together and cerebras: streaming and non-streaming both produce thinking blocks, tool_use is still emitted, multi-turn replay of a thinking block returns 200, and the stop_sequences classifier path is unchanged. Also confirmed in real Claude Code, which renders the reasoning as a "thought for Ns" section.

Known limits

• Interleaved reasoning-after-text yields [thinking, text, thinking, text], which violates Anthropic's ordering rule. Not guarded: none of the 26 model×provider combinations tested interleaves — reasoning always arrives wholly before text.
• A tight max_tokens on a reasoning model can now return a thinking-only response with no text block, because the budget is spent on reasoning. Previously the same call returned truncated text.

🤖 Generated with Claude Code