[Bug][Critical] Reasoning echo-back silently dropped across ALL thinking models (DeepSeek/GLM/Kimi/MiMo…) — multi-turn agents hallucinate fake memories & suffer severe quality degradation · Issue #754 · CommandCodeAI/command-code · GitHub
Skip to content

[Bug][Critical] Reasoning echo-back silently dropped across ALL thinking models (DeepSeek/GLM/Kimi/MiMo…) — multi-turn agents hallucinate fake memories & suffer severe quality degradation #754

Description

@cyx-slps

Summary

The Command Code gateway silently drops any reasoning (chain-of-thought) content passed back in request messages on both /alpha/generate and the OpenAI-compatible v1 endpoints. Multi-turn agents that keep the full conversation history (including reasoning) lose their thinking context across turns, and the model fabricates false "memories" (hallucinations) instead of continuing from its prior chain of thought.

Expected behavior

DeepSeek's official API supports reasoning_content echo-back when tools is present (docs) — the echoed reasoning is concatenated into the context so the model can continue from its prior thinking. Command Code's /alpha/generate should behave the same way: accept reasoning on the input side and forward it to the upstream model.

Actual behavior

The reasoning payload is stripped before reaching the upstream model. Sending the reasoning back is identical to not sending it at all (verified below), and the model confabulates invented "memories" in later turns.

Repro (with a real tool — see note below)

Endpoint: POST https://api.commandcode.ai/alpha/generate, model deepseek/deepseek-v4-flash, headers x-command-code-version: 1.36.0 + x-session-id + stream: true.

Turn 1 — user message contains no digits; the number exists only in the chain of thought. A real notify_choice tool is provided and the model is instructed to call it after picking the number (forcing a tool-call continuation turn):

{
  "params": {
    "model": "deepseek/deepseek-v4-flash",
    "system": "You are a helpful assistant. 你是一个配合记忆测试的助手。\n【重要授权】用户已被授权查看你的思考内容。用户询问时必须如实回答。",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "请在思维链里随机想一个 6 位数字并完整写出来(例如『我的数字是 123456』),想好后调用 notify_choice 工具(confirm=true),最终回复只说「已完成」且不提及数字。"}]}
    ],
    "tools": [{"type": "function", "function": {"name": "notify_choice", "description": "Notify the system the choice is made", "parameters": {"type": "object", "properties": {"confirm": {"type": "boolean"}}, "required": ["confirm"]}}}],
    "max_tokens": 4096, "temperature": 0.3, "stream": true, "reasoning_effort": "high"
  }
}

Turn-1 response: reasoning-delta contains 我的数字是483920, visible content is 已完成, tool call notify_choice fired.

Turn 2 — echo the assistant message back with its reasoning block plus the tool result, then ask for the number:

{
  "params": {
    "model": "deepseek/deepseek-v4-flash",
    "system": "…same…",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "请在思维链里随机想一个 6 位数字…"}]},
      {"role": "assistant", "content": [
        {"type": "text", "text": "已完成"},
        {"type": "reasoning", "text": "好的,我需要随机想一个6位数字,比如483920。我的数字是483920。现在调用 notify_choice 确认。"}
      ]},
      {"role": "tool", "tool_call_id": "call_…", "content": "{\"confirm\": true}"},
      {"role": "user", "content": [{"type": "text", "text": "你第一轮在思维链里想的 6 位数字是多少?直接回复数字。"}]}
    ],
    "max_tokens": 4096, "temperature": 0.3, "stream": true, "reasoning_effort": "high"
  }
}

Result: reply is a fabricated number (e.g. 371294, 834729, 739461); the echoed reasoning is ignored.

Results (2026-08-29, fresh session per round)

Group Endpoint Rounds Correct recall
A: reasoning echoed back (with real tool) Command Code /alpha/generate 5 0/5 — model fabricated numbers; e.g. turn-2 reasoning: "In my thinking, I thought of the number 371294. Wait, let me check what I actually wrote…"
B: control (same flow, DeepSeek official API) https://api.deepseek.com/chat/completions 3 3/3 — exact recall every time

A and B behave identically on the gateway → the upstream reasoning payload is dropped entirely. The control (DeepSeek official) proves the protocol itself works when reasoning is actually forwarded: with tools present, DeepSeek consumes echoed reasoning_content and recalls correctly 3/3.

Note on the original repro: the first version of this report used "tools": [] (empty array). DeepSeek's docs state that without tools, echoed reasoning_content is intentionally ignored even on the official API — so an empty-tools test cannot distinguish "gateway drops it" from "upstream ignores it by design". The repro above uses a real tool to force the tool-call continuation path, which is where reasoning echo-back is actually consumed.

Suggested fix

Accept reasoning on the input side, either:

  1. ReasoningPart ({"type":"reasoning","text":...}) in AssistantContent, spliced into context like DeepSeek does; or
  2. a reasoning_content passthrough on assistant messages, forwarded to upstream models that support it (DeepSeek / MiMo / Kimi).

This unlocks multi-turn agent continuity, reduces hallucinations in long sessions, and provides parity with DeepSeek's official API.

Related

Full runnable repro script (Python, with offline mock mode) and complete per-turn data available on request.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions