[Bug][Critical] OpenAI multimodal content ARRAY (text parts) on assistant tool-call messages is silently DROPPED ENTIRELY — string content survives; agents lose visible-text context across tool loops · Issue #806 · CommandCodeAI/command-code · GitHub
Skip to content

[Bug][Critical] OpenAI multimodal content ARRAY (text parts) on assistant tool-call messages is silently DROPPED ENTIRELY — string content survives; agents lose visible-text context across tool loops #806

Description

@cyx-slps

Summary

The CommandCode gateway silently drops the entire content array of assistant messages that carry tool_calls, when that content uses the OpenAI multimodal parts format (content: [{type:"text", ...}, ...]). The model then sees no text at all from that turn — only the tool call. The same text sent as a plain string content survives 100%.

This is broader than #754 (reasoning fields dropped): here even ordinary visible text is lost, purely because of the wire shape (array vs string). Any client that replays assistant history as content-parts arrays (OpenAI SDK structured/multimodal paths, Responses→Chat bridges, agent frameworks that normalize content to parts) silently loses visible-text context on every tool-call turn → multi-turn memory breaks and models confabulate.

Repro

POST https://api.commandcode.ai/provider/v1/chat/completions, model deepseek/deepseek-v4-flash, streaming. History contains an assistant message with tool_calls whose content is a two-part text array carrying unique probe words (蓝鲸 / 海豚):

{
  "model": "deepseek/deepseek-v4-flash",
  "messages": [
    {"role": "user", "content": "请回复『好的』两个字即可,不要调用工具。"},
    {
      "role": "assistant",
      "content": [
        {"type": "text", "text": "标记词A:蓝鲸"},
        {"type": "text", "text": "标记词B:海豚"}
      ],
      "tool_calls": [
        {"id": "call_probe_1", "type": "function",
         "function": {"name": "notify_choice", "arguments": "{\"ok\": true}"}}
      ]
    },
    {"role": "tool", "tool_call_id": "call_probe_1", "content": "{\"ok\": true}"},
    {"role": "user", "content": "上一轮 assistant 消息内容里有两个标记词,分别是什么?请列出你看到的全部标记词,不要编造。"}
  ],
  "tools": [{"type": "function", "function": {"name": "notify_choice", "description": "通知系统你已完成阅读", "parameters": {"type": "object", "properties": {"ok": {"type": "boolean"}}, "required": ["ok"]}}}],
  "stream": true, "reasoning_effort": "high", "temperature": 0
}

Control: identical payload but content as a string — "标记词A:蓝鲸\n\n标记词B:海豚".

Results (2026-09-06, fresh sessions)

Wire shape of assistant content Probe recall
content array of 2 text parts (+ tool_calls) 0/3 — model: "上一轮 assistant 消息里没有任何文字内容,只包含一次工具调用(notify_choice)"
content string with same text (+ tool_calls) 3/3 — both probe words recalled exactly

Analysis

The drop is all-or-nothing on the array, not per-part: neither part survives. This matches the observation in #754's comment thread that {type:"text"} parts are "accepted but dropped from context when tool_calls are present" — this report isolates it as a standalone, reproducible bug with a minimal probe.

Likely same root cause as #754: the gateway's OpenAI→(Vercel ModelMessage?) serialization for assistant messages consumes tool_calls but discards text content when it arrives as a parts array, while the string form takes a different (lossless) path. Fixing the serializer to spread array text parts into the outbound message fixes both #754-style reasoning replay and this plain-text loss.

Impact

  • Any agent that serializes assistant history with content-parts arrays loses all visible text of tool-call turns — not just reasoning — causing hallucinations on later turns.
  • Silent: HTTP 200, no error, no warning; only observable via recall probes.
  • Workaround today: always flatten assistant content to a single string before replay.

Related

Probe script (Python, offline-runable) available on request.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions