feat: get_final_result() on the Messages stream, and a cut managed answer raises by rejojer · Pull Request #607 · VectifyAI/PageIndex · GitHub
Skip to content

feat: get_final_result() on the Messages stream, and a cut managed answer raises - #607

Merged
rejojer merged 8 commits into
mainfrom
fix/messages-final-result
Oct 21, 2026
Merged

rejojer merged 8 commits into
mainfrom
fix/messages-final-result

Conversation

@rejojer

@rejojer rejojer commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

chat(protocol="messages", stream=True) now gives the run's totals through stream.get_final_result() instead of a closing result event, so the stream itself carries only the Anthropic SDK's events, exactly as in 0.4.4. Answers that stop short raise on more paths instead of passing as complete: a managed (cloud) chat() cut at max_tokens or refused, an answer cut by a full context window on the cloud or claude-code/, and the text stream of chat_completions(stream=True). On claude-code/, an answer the CLI resumed after its output cap no longer loses its first part, and a stream the CLI retried no longer repeats its text.

Changes

  • get_final_result() replaces the result event (feat: report token usage for local indexing and chat runs, failures included #572, unreleased). The stream is now a MessagesStream. Iterating it yields the Anthropic SDK's events, so a finished run ends with message_stop again and code reading events[-1].message keeps working. get_final_result() reads the rest of the stream first, then returns the run's usage (summed across turns), its messages turn sequence and its final turn's stop_reason, the same values the non-stream envelope carries, so a caller reading only the final result can still tell a cut answer. Called before the end, it waits for the run rather than return a partial total, and a failed run raises the same error from it as iterating does. This follows the official SDKs: Anthropic's get_final_message(), OpenAI's get_final_response(), the tool runner's until_done(). MessagesStream is exported next to ChatStream.
  • A cut managed answer raises. The managed endpoint reports Anthropic's stop reasons as finish_reason ("max_tokens", "refusal"), not OpenAI's "length" and "content_filter", so the answer lane's cut check never fired on the cloud. The check now knows both sets of names: chat() raises with the text in partial_answer, and a stream raises after its text, as own-model chat already does. The endpoint's values pass through untouched: protocol="chat_completions" still returns finish_reason as the endpoint sent it.
  • An answer cut by a full context window raises too. Claude stops with stop_reason "model_context_window_exceeded" when its context window fills before the answer is finished, and on claude-code/ and the managed endpoint such an answer passed as complete. It now raises as a max_tokens cut does, and protocol="chat_completions" on claude-code/ marks it "length". The Claude Code CLI (2.1.301) builds the same max_output_tokens error for this stop reason that it builds for max_tokens. LiteLLM-routed Claude models still get "stop", since LiteLLM maps this stop reason that way.
  • chat_completions(stream=True)'s text stream raises on a cut answer. Without stream_metadata it yields only the answer text, and on every engine it dropped the finish chunk, so an answer cut at max_tokens, filtered, or (on the managed endpoint) refused ended like a finished one, though this stream is documented as chat(stream=True, show_process=False), which raises. It now raises after the text, with the run's usage. stream_metadata streams keep marking the cut in finish_reason. What the stream yields is unchanged.
  • A resumed claude-code/ answer comes back whole. When an answer hits the Claude Code CLI's output cap, the CLI resumes it with a second model call, but its ResultMessage.result holds only the last text block, so chat() and chat_completions() returned only the last part, as if complete, while the stream showed both. The answer is now the run's own assistant text since the last tool call.
  • A retried claude-code/ stream doesn't repeat its text. When a stream failed after some text and the CLI retried the model call, the retry's text went out a second time. A retry's text now goes out only once it passes what was already sent; a retry that writes it differently follows a blank line, as the CLI's non-streaming fallback already did.
  • Claude's refusal stop maps as LiteLLM maps it. The claude-code/ engine mapped only max_tokens to an unfinished finish_reason; it now maps Claude's stop reasons the way LiteLLM's finish-reason table does (max_tokens to "length", refusal to "content_filter"). Today's Claude Code CLI (2.1.299 to 2.1.301) reports a refusal as an API error instead, which claude-code/ already raises as Claude Code failed: ... with usage; the mapping covers a CLI that reports the stop reason, as the max_tokens branch does.

Verification

  • CI: 970 passed with frameworks on py3.10 and py3.13 (969 + 1 skipped on pdfium 4), 689 passed + 281 skipped without, on anthropic 1.17.0, claude-agent-sdk 0.2.170, openai-agents 0.20.0 and litellm 1.105.3.
  • Locally: 963 passed + 7 skipped; with the frameworks blocked, 682 passed + 288 skipped. The first commit passes on its own too, and the Messages tests pass on anthropic 0.122.0, the floor.
  • The real bundled Claude Code CLI (2.1.301), run against a local mock Messages API: an answer resumed after the output cap, a stream dropped and retried, and a reply in two text blocks all come back whole and once, from chat(), its stream and chat_completions(); with the resumes used up, all three still raise. A refused answer streams a message_delta with stop_reason "refusal" before the CLI turns it into an error, and that is the stop reason the managed endpoint relays as finish_reason. The CLI version the managed endpoint runs (2.1.273), read the way the endpoint reads it, ends a refusal with "refusal" after its text and an answer still cut after three resumes with "max_tokens".
  • The new tests catch seventeen mutants: no drain before returning, the captured result overwritten, the Anthropic names dropped, the claude-code/ refusal mapping dropped, "model_context_window_exceeded" dropped from either list (two), ResultMessage.result used again, the tool-call reset dropped, every retried delta sent again, the fallback compared with the retry's own text, a differing retry sent without its blank line, the text stream's raise dropped, each engine's text stream left on its old path (three), no stop_reason, the first turn's stop_reason.

chat(protocol="messages", stream=True) now returns a MessagesStream.
Iterating it yields the Anthropic SDK's events and nothing else, as in
0.4.4, so a finished run ends with message_stop again and code reading
events[-1].message keeps working. Its get_final_result() reads the rest
of the run first, then returns the run's usage (summed across turns)
and its messages turn sequence: the values the non-stream envelope
carries. A failed run raises the same error from it as iterating does;
after a failure or close() it raises rather than return nothing.

This replaces the "result" event #572 appended to the stream, which
never shipped. The Anthropic and OpenAI SDKs hand a stream's final
value to a method that drains the stream first (get_final_message(),
get_final_response(), the tool runner's until_done()), so a read made
before the end can never return a partial total, and the event
sequence stays the tool runner's own.

MessagesStream is exported next to ChatStream, for annotations.
The managed endpoint reports Anthropic's stop reasons as finish_reason
("max_tokens", "refusal"), not OpenAI's "length" and "content_filter",
so the answer lane's cut check never fired on the cloud: chat()
returned a truncated answer as if it were complete. The check now knows
both sets of names, so a managed chat() raises on a cut answer with the
text in partial_answer, and a managed stream raises after its text, as
own-model chat already does.

The endpoint's values are passed on untouched: protocol=
"chat_completions" still returns finish_reason as the endpoint sent it.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Comment thread pageindex/client.py
The claude-code/ engine mapped only Claude's max_tokens stop to an
unfinished finish_reason; every other stop, refusal included, would
read as a finished answer. It now maps Claude's stop reasons the way
LiteLLM's finish-reason table does: max_tokens to "length", refusal to
"content_filter". The answer lane raises on both, with the text in
partial_answer, and a stream raises after its text; chat_completions
marks the finish_reason.

Today's Claude Code CLI (2.1.299 to 2.1.301, bundled with
claude-agent-sdk 0.2.166 to 0.2.168) never reports these stops this
way: it turns a refusal, like a max_tokens cut, into an API-error
result, which claude-code/ already raises as "Claude Code failed: ..."
with usage and no partial_answer. The mapping covers a CLI that
reports the stop reason instead, as the max_tokens branch does.
… resume

A claude-code/ answer that hit the CLI's output cap came back from
chat() and chat_completions() as its last part only, as if complete.
The CLI resumes such an answer with a second model call, but its
ResultMessage.result, which the engine returned, holds only the last
text block; the stream already showed both parts. Reproduced with the
real bundled CLI (2.1.301) against a local mock API: the stream gave
"Part one of the answer, and part two.", chat() gave "and part two.".
A reply split into two text blocks lost its first block the same way.

The engine now builds the answer from the run's own assistant text:
everything since the last tool call, so a note written before a tool
call is still left out, as result left it out.

Three tests faked a run as a ResultMessage alone; they now send the
assistant message first, as the CLI does.
When a claude-code/ stream failed after some text and the CLI retried
the model call, the retry streamed the answer again from its start and
every delta went out a second time: the stream read "Part one of the
answer, Part one of the answer, and part two." Reproduced with the real
bundled CLI (2.1.301) against a local mock API whose stream dropped
after the first text block. The engine reset its record of the streamed
text on each message_start, so the dedup that guards the CLI's
non-streaming fallback forgot what the caller already had.

It now keeps two records: what went out to the caller since the last
text block (kept across a retry), and this model call's own deltas. A
retry's deltas go out only once they pass the text already sent; a
retry that writes it differently follows a blank line, as the fallback
does. The fallback compares its whole text against what went out.

The test row that pinned the repeated text now expects it once; two
rows add a retry cut sooner than the first try, then the fallback, and
a retry that regenerates differently.
chat_completions(stream=True) without stream_metadata yields only the
answer text, and on every engine it dropped the finish chunk: an
answer cut at max_tokens, filtered, or (on the managed endpoint)
refused ended like a finished one, though chat_completions documents
this stream as chat(stream=True, show_process=False), which raises
after the text.

All three engines now read the stream through one helper that yields
the text, then raises the answer lane's cut error, with the run's
usage, when the finish_reason says the answer stopped short. The
managed engine's own text-only parser is gone: the helper reads its
full chunk stream, and reads a chunk's delta defensively, as that
parser did. What the stream yields is unchanged; stream_metadata
streams keep marking the cut in finish_reason, as the protocol does.

The managed endpoint does send such a finish_reason. It relays the
stop_reason of the last message_delta its Claude Code run streamed and
does not check the run's error flag. A refusal streams a message_delta
with stop_reason "refusal" before the CLI turns it into an API-error
result, so the stream ends with finish_reason "refusal" and [DONE]
after whatever text came first; an answer still cut at max_tokens once
the CLI's three resumes run out ends with "max_tokens" the same way.
Run against a local mock API with the CLI the endpoint pins (2.1.273,
claude-agent-sdk 0.2.153) and with 2.1.301.
MessagesStream.get_final_result() returned the run's usage and messages
only. A caller that read the answer from it, as from the official SDK's
get_final_message(), could not tell an answer cut at max_tokens (or a
run stopped at max_turns on a tool call) from a finished one; the
non-stream envelope carries stop_reason for exactly that.

The final result now also carries the last turn's stop_reason, the
value the non-stream envelope reports.
@rejojer
rejojer force-pushed the fix/messages-final-result branch from 6957b1c to 7a114b4 Compare October 21, 2026 11:43
Claude stops with stop_reason "model_context_window_exceeded" when the
context window fills before the answer is finished. The answer lane
counted only max_tokens and refusal as cut, so chat() returned such an
answer as complete, on claude-code/ and on the managed endpoint, which
relays the stop reason as its finish_reason. The Claude Code CLI
(2.1.301) builds the same max_output_tokens error for this stop reason
that it builds for max_tokens.

It now raises like a max_tokens cut. The chat_completions protocol
lane on claude-code/ marks it "length", as it marks max_tokens.

LiteLLM-routed Claude models are unchanged: LiteLLM maps this stop
reason to "stop".
@rejojer rejojer closed this Oct 21, 2026
@rejojer rejojer reopened this Oct 21, 2026
@rejojer rejojer closed this Oct 21, 2026
@rejojer rejojer reopened this Oct 21, 2026
@rejojer rejojer closed this Oct 21, 2026
@rejojer rejojer reopened this Oct 21, 2026
@rejojer rejojer closed this Oct 21, 2026
@rejojer rejojer reopened this Oct 21, 2026
@rejojer
rejojer merged commit ce3b27f into main Oct 21, 2026
32 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant