provider: big-pickle leaks chain-of-thought into visible text when reasoning contains the literal close-think tag · Issue #55268 · anomalyco/opencode · GitHub
Skip to content

provider: big-pickle leaks chain-of-thought into visible text when reasoning contains the literal close-think tag #55268

Open

Description

@hollychan05-ctrl

Summary

On opencode/big-pickle, when the model emits the literal 8-character closing tag
</think​> inside its own reasoning, the provider behaves as if the reasoning
has ended: the remainder of the chain-of-thought is streamed on the visible
content channel instead of reasoning_content. opencode persists that text as an
ordinary text part, so the user sees raw chain-of-thought spliced into the reply,
and the reasoning part stops mid-sentence.

This looks like a provider / generation-stop-string issue triggered by the model's
own output, not a renderer issue: the two channels are split at the exact byte
offset of the tag.

Environment

  • opencode version: 2.0.28 (release channel)
  • OS: Windows 11 (10.0.26200, x64)
  • Terminal: Unavailable: terminal environment variables are not set
  • Shell: C:\Windows\system32\cmd.exe
  • Install/channel: latest
  • Active plugins: opencode-tavily, @firecrawl/opencode
    (+ local plugins think-guard, smart-router, vision-router under ~/.config/opencode/plugins/)
  • Model: opencode/big-pickle (variant default), agent build

Reproduction

  1. Start a session with opencode/big-pickle (default variant).
  2. Put the literal closing think tag in the conversation context — e.g. ask the
    model to discuss or quote reasoning-model tags, or simply have the model's own
    visible text quote it once. (A prompt that never mentions the tag does not
    trigger it.)
  3. Ask anything reasoning-heavy.
  4. The stream splits: thinking runs in the reasoning channel for a while, then
    jumps to the content channel in the middle of a sentence and continues the
    same chain-of-thought as visible text.

Controlled experiment (same model + agent):

  • Prompt that contains the literal tag → the leak reproduces reliably.
  • Prompt that never mentions the tag → reasoning_content and content stay
    cleanly separated, no leak.

Expected Behavior

All chain-of-thought stays on the reasoning channel (reasoning_content) and is
rendered as a collapsible thinking block; the visible content contains only the
answer. Emitting the literal tag inside reasoning must not terminate the
reasoning channel.

Actual Behavior

Raw SSE is split mid-sentence between the two channels. Captured from the
provider response stream (choices[0].delta), lightly trimmed:

... delta.reasoning_content: "... the model writes the 8-character sequence `"
... delta.content:           "` literally at least twice. Let me count: ..."

The backtick at the end of reasoning_content and the backtick starting
content are one and the same character — the JSON string was cut across the two
fields at the tag boundary.

The same split is visible in the local session DB
(…/opencode.db, session_message, type='assistant' → data.content): a
reasoning part that ends mid-sentence, immediately followed by a text part
that continues that same sentence, e.g.

{
  "content": [
    { "type": "reasoning", "text": "…BUT countTagsOut=1 — coun" },
    { "type": "text", "text": "? not present. \"</think​>\"? not present (there's a ZWSP). So count should be 0 …" }
  ]
}

The leaked text can be very large (worst case observed: a 34,404-character
text part against a 69-character reasoning part in the same message).

Additional Context

Trigger

The trigger is the literal close tag appearing in the model's context (which
causes the model to reproduce it while reasoning). Controlled comparison with the
same model/agent/prompt shape:

prompt mentions the literal </think​> result
no reasoning_content and content cleanly separated
yes reasoning cut mid-sentence, remainder continues on content (leak)

Timeline (one affected session, local DB)

time (UTC) result
17:09–17:35 frequent leaks (e.g. reasoning 69 / text 34,404)
17:39:05 last leak: reasoning ends "…countTagsOut=1 — coun", text starts "? not present. …"
17:39:44 → no further leaks across ~15 turns once the trigger was removed from the outgoing context (see workaround)

tokens.reasoning is not a reliable signal: it is often 0/absent on leaked
turns and also 0/absent on some healthy turns.

Related observation (possibly the same root cause)

Separately, a small session-clustered set of opencode/big-pickle turns persist
tokens.reasoning > 0 but contain no reasoning part and a clean text
part (aggregate over ~2.7k assistant messages: 2,754 with tokens.reasoning > 0,
of which 61 have no reasoning part and no tag in text). The TUI renders no
thinking block for those turns. This may share the same provider-side stop-string
handling; the leak above is the case with direct raw-SSE evidence.

Workaround (client side)

Neutralising the literal tag in the outgoing context before it reaches the model
prevents the trigger. Inserting a zero-width space into the token — so the model
sees the same visual text but not the exact byte sequence — eliminated the leak
across subsequent turns in a session that had been leaking heavily:

</think​>  ->  </think​>      (U+200B inserted)

Doing this in every string of the outgoing message history (user, assistant, tool
results, system) removes the trigger. The complete fix, however, is provider-side:
a model-emitted tag should not terminate reasoning / switch the delta channel.

Raw closing-tag bytes: 3c 2f 74 68 69 6e 6b 3e.

Activity

  1. hollychan05-ctrl commented on Oct 14, 2026

    @hollychan05-ctrl
    Author

    Copy/paste note for the reproduction: the closing tag is exactly the 8 bytes hex 3c 2f 74 68 69 6e 6b 3e (less-than, slash, t h i n k, greater-than).

    The tag renderings elsewhere in this report are visually identical but may carry an invisible zero-width character depending on how they were copied. If the reproduction does not trigger, retype the tag from those bytes (in JS: "\u003c/think\u003e").

  2. hollychan05-ctrl commented on Oct 14, 2026

    @hollychan05-ctrl
    Author

    Update (opencode 2.0.29): the context-side workaround is not sufficient in tag-heavy sessions — the model reconstructs the tag and still splices

    New evidence from a session where the client-side defuse was active the whole time (context defused {"tags":N} fired on every turn, i.e. every literal tag in the outgoing messages was neutralised with an inserted U+200B).

    Session ses_ec5e33619ffeyWCNsuzx9N6pKp, model {"id":"big-pickle","providerID":"opencode"}, opencode 2.0.29.

    From opencode.db → session_message (type='assistant' → data.content):

    seq time (UTC) reasoning len text len observation
    165 11:15:29 1959 4973 text starts mid-reasoning: ` inside its reasoning; the provider treats it as end-of-reasoning and routes the *rest of…
    180 11:15:51 411 6941 reasoning ends …模型在 thinking 裡吐出字面 ` and text starts `;供應商把之後的 thinking 切到 content 通道…
    192 11:17:02 806 6940 reasoning ends mid-word; text continues the same chain-of-thought

    In 180/192 the leaked fragment is prepended to the legitimate answer inside the same text part (the real answer is intact after the fragment).

    Why the workaround no longer holds here: the trigger is no longer the incoming context but the model's own generated reasoning. Because the session itself discusses the tag (its plugin source, logs and README are in context), the model reconstructs the canonical bytes (3c 2f 74 68 69 6e 6b 3e) in its reasoning and emits them, so defusing the outgoing messages cannot prevent it. This confirms that the context workaround is a mitigation only and raises the priority of the provider-side fix: a model-emitted close tag must not terminate the reasoning channel / switch the delta channel.

    Also note the stream-side net is not a substitute: when active it drops leading characters of otherwise legitimate answers (rewrite {"event":"cleaned","dropped":23} observed).

    Environment: opencode 2.0.29; Windows 11 (10.0.26200, x64); shell cmd.exe; provider/model opencode/big-pickle.

  3. hollychan05-ctrl commented on Oct 14, 2026

    @hollychan05-ctrl
    Author

    Update: wire-level capture of the second form (missing opening tag) — the client cannot recover from it

    Same reporter, same machine. Since my original report I captured the raw provider SSE while a session was actively leaking, and it shows a second failure mode that is worse than the tag-splice one because nothing is left for the client to detect.

    Environment

    • opencode 2.0.29 (was 2.0.28 in the original report; the leak survived the upgrade)
    • Windows 11 10.0.26200 x64, cmd.exe
    • Model opencode/big-pickle (default variant), agent build
    • Capture: an http.response hook on responses coming from providerID: "opencode" that tees the SSE body and logs the raw chunks (no rewriting)

    Wire evidence (live leaking session, 2026-10-14T12:14Z)

    Self-contained excerpt — this is the complete response of one assistant step (generated id style router-…, 6 data: lines):

    {"delta":{"role":"assistant","content":"","reasoning_content":null}}
    {"delta":{"role":null,"content":"</think\u200b>","reasoning_content":null,"tool_calls":null}}
    {"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"index":0,"id":"…","function":{"name":"…","arguments":"…"}}]}}
    {"delta":{"reasoning_content":null},"finish_reason":"tool_calls"}
    {"choices":[],"usage":{"prompt_tokens":43656,"completion_tokens":80,"total_tokens":43736,"prompt_tokens_details":{"cached_tokens":43008}}}
    

    Nine consecutive captured responses from that session, all identical in shape:

    time (Z) data lines bytes on content bytes on reasoning_content opening tag in content closing tag in content
    12:14:03 6 8 0 no yes
    12:14:11 123 ≥1200 (probe cap) 0 no no
    12:14:14 46 427 0 no yes
    12:14:17 6 8 0 no yes
    12:14:23 199 ≥1200 0 no no
    12:14:49 474 ≥1200 0 no yes
    12:14:50 6 8 0 no yes

    Points that matter:

    1. delta.reasoning_content is present as a key in every chunk but is always null. Chain-of-thought is streamed on delta.content.
    2. The opening tag never appears in content. The closing tag does (and often arrives as its own isolated chunk — 8 bytes — immediately before a tool_calls chunk).
    3. usage.reasoning_tokens / opencode's tokens.reasoning is 0 for these messages, so the client-side token accounting agrees with the provider: as far as the API is concerned, no reasoning was produced.

    Same probe, same model, ~19h earlier: reasoning_content was populated

    On 2026-10-13T17:28–17:44Z the same hook captured 51 responses from opencode/big-pickle on this machine; 24 of them carried reasoning text:

    17:29:25 | dataLines=112  | content=173 bytes | reasoning=900 bytes (probe cap)
    17:29:59 | dataLines=1376 | content=900       | reasoning=900
    17:30:21 | dataLines=2217 | content=900       | reasoning=900
    17:43:50 | dataLines=1129 | content=502       | reasoning=900
    

    So reasoning_content can and did carry the CoT for this exact model through this exact gateway. Today it never does. That asymmetry is on the provider/gateway/generator side — by the time the response reaches opencode there is nothing to separate.

    Why this cannot be fixed in the client

    opencode splits reasoning from visible text purely on the API fields (reasoning_content / reasoning / reasoning_details). There is no tag-based fallback: the byte sequence </think does not occur anywhere in opencode.exe. When the provider reports no reasoning field and no reasoning tokens, opencode stores the whole string as a text part and renders it — there is no local signal left to key on.

    DB-level shape of the two forms

    Both were observed in the same install (session rows in session_message, data JSON):

    form reasoning part tokens.reasoning visible text part
    A — tag splice (original report) present, ends mid-sentence > 0 (e.g. 141, 186, 653) CoT remainder + literal </think\u200b> + answer
    B — missing opening tag (this comment) absent 0 whole CoT + trailing </think\u200b>

    Form B now dominates:

    • Session ses_ec5fafc6… ("performance triage"): 94 of 99 assistant turns have no reasoning part; the leak is present from the second assistant turn onward and still present at the 99th, ~65 minutes later.
    • Session ses_ec5e336… (this report's own triage session): the first 45 assistant turns all had a reasoning part; then from 11:56:43Z 15+ consecutive turns have none, with individual text parts of 8k–22k characters of raw CoT.

    It is self-reinforcing

    Once the stored history contains assistant turns whose text is "CoT … </think\u200b>", every later turn reproduces that template: the model stops emitting the opening tag, so the provider never opens a reasoning channel, so the next turn is stored the same way. Both sessions show exactly this: clean first turns → then permanently form B, surviving session restarts (and a 2.0.28 → 2.0.29 upgrade). It is not caused by plugins — the split happens at the wire, upstream of any context/http.response hook, and the same capture shows the provider's bytes before the client persists anything.

    Suggested fix direction

    • Provider/gateway: do not gate the reasoning channel on an opening tag the model may omit, and do not split a reasoning stream at a tag occurrence that the model can legitimately write inside its reasoning. If the model ends its reasoning with </think…>, the text before it is still reasoning.
    • opencode (defensive): if a whole assistant message arrives with no reasoning_content and reasoning_tokens == 0, but content contains a closing think tag (tolerant of the Unicode zero-width suffix characters, as plugins already have to be), treat the prefix as reasoning instead of visible text. That would at least stop form B from being persisted and re-fed as visible history.

    Related reports: #54586 ("model reasoning always visible, because starttag is missing", closed as not planned), #54599, #54886.

  4. hollychan05-ctrl commented on Oct 14, 2026

    @hollychan05-ctrl
    Author

    Addendum: the splice point of form A is an inline-code quotation of the tag

    Reconstructed from the persisted parts (session_message.data.content[]). Form A — a reasoning part that stops mid-sentence with the remainder visible — does not cut at an arbitrary byte. It cuts exactly where the model wrote the tag inside inline code, i.e. `</think…>`.

    Reasoning tail vs. visible-text head, verbatim (several sessions, 2.0.28 and 2.0.29):

    R: ... the model emits the literal 8-char `
    T: ` inside its reasoning; the provider ...
    R: ... 根因: 模型在 thinking 裡吐出字面 `
    T: `;供應商把之後的 thinking 切到 content 通道 ...
    R: ... everything until a `
    T: ` is discarded. Since leak form is ...
    R: ... reasoning continued: "...the literal `
    T: ` tag — yes! In my answer I wrote ...
    

    So the model's stream is

    <CoT> … the literal 8-char `</think​>` inside its reasoning … </CoT><answer>
                              ^^^^^^^^^^^ the byte sequence the provider matches on
    

    and the provider behaves as if reasoning ended at that offset:

    1. the reasoning part keeps everything up to the tag (so it ends right after the opening backtick, mid-sentence);
    2. the 8 tag bytes are consumed — they do not appear on content;
    3. the remainder is routed to the visible channel, which therefore starts with the closing backtick of the quotation.

    That last point is a useful contrast with form B (the reasoning_content: null form from my previous comment): there the tag is not consumed — my wire capture literally shows {"delta":{"content":"</think\u200b>"}}, i.e. the provider forwards those bytes verbatim when it is not in reasoning state. So the same byte sequence is treated as a control token in one state and as ordinary text in the other.

    How often

    Per session, counting assistant messages that have a reasoning part, how many end mid-sentence, and how many of those end at a backtick:

    session msgs with reasoning cut mid-sentence of those, cut at a backtick
    session A 51 5 4
    session B 24 2 2
    session C 7 2 1
    session D 15 1 1
    session E 5 1 1

    (The same sessions have the tag sitting in visible text parts in 14–66 messages each.)

    Implication

    The reasoning channel is terminated by the model's own output, and the trigger is any mention of the tag — a bug report like this one is the worst case, because the tag is then quoted in inline code almost by construction. Nothing on the client side can prevent it: by the time the response is persisted, the reasoning part is simply truncated and the remainder is ordinary text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions