Give each chat stream failure a stable code so the interface words it in the reader's language by Maya-Kid · Pull Request #951 · deeplethe/utopia · GitHub
Skip to content

Give each chat stream failure a stable code so the interface words it in the reader's language - #951

Merged
WaylandYang merged 1 commit into
deeplethe:devfrom
Maya-Kid:fix/chat-stream-error-codes
Sep 26, 2026
Merged

WaylandYang merged 1 commit into
deeplethe:devfrom
Maya-Kid:fix/chat-stream-error-codes

Conversation

@Maya-Kid

Copy link
Copy Markdown
Contributor

Problem

ADR 0004 makes one hard rule: the server produces no display text. An error a user can hit carries a code, and the interface words it from its err.* table. The English message stays for MCP, CLI and logs. Refused requests follow this rule, and #931 gave the chat request the same wording.

Failures inside the chat stream do not follow it. The error frame is a plain English sentence, and the web client shows it as it is. In the Chinese interface, these all appear in English:

  • an account out of credit;
  • a rate limit;
  • an empty answer;
  • a failed save.

Change

  • Server (api/chat.rs):
    • A failed generation carries a Failure { code, message } instead of a string.

    • The error frame's data is the same envelope as a refused request: {"error": <message>, "code": <code>}. The English message is unchanged.

    • The codes:

      code when
      answer_empty the model returned nothing (loop and one-shot fallback)
      answer_tool_text a tool call written as text: as the budget answer, or twice in an ordinary turn (Keep a tool call written as text out of the chat answer in every turn, and send the turn back once #946)
      answer_tool_call the model called a tool after the budget ran out
      answer_too_long the budget answer passed the 1 MiB bound
      answer_failed the final answer failed for another reason, the stream ended without a result, or the producer misbehaved
      answer_not_saved the answer could not be saved, or its save could not be confirmed
      search_failed the one-shot fallback could not search the documents
      stream_ended, stream_lagged the live connection ended early, or fell behind the broadcast
      model_out_of_credit, model_rate_limited, model_unavailable, model_unreachable, model_rejected the model endpoint failed
    • Model failures take their code from the types utopia_llm already has: OutOfCredit, RateLimited, Unavailable, Unreachable or Interrupted, and Rejected. Anything else is answer_failed.

    • The finalization reasons from agent::finalization_error reach chat_finalization as text, so unpublishable_code reads them by their wording. A unit test builds the reasons with finalization_error itself, so rewording one fails that test instead of silently losing its code.

  • Web (api.ts):
    • An error frame is read with the same refusalMessage as a refused request.
    • Data that is not JSON is shown as it is, so frames from an older server still read.
  • Wording: the new codes are added to err.* in en.ts and zh.ts. zh is typed typeof S, so a missing entry fails the build.
  • Docs: the status line of 0004 now says that the stream's error frames carry the same code.

Relation to #946

#946 added one more failure exit in the same place: an ordinary turn that writes its tool call as text a second time. This branch is rebased onto dev after #946 and codes that exit answer_tool_text, and the new tests check it.

Verification

Run in containers on a Linux host (Rust 1.98.1, Postgres 16 with pgvector; Node 20 with pnpm 10.2.1).

  • Terminal contract (chat_terminal_tests.rs):

    • The rule "the error states a reason" is tightened to "the error frame is JSON with a non-empty code and message".
    • Every failure row in that table now checks for a code.
  • New tests in chat_error_code_tests.rs run through the real chat handler against a scripted endpoint:

    • an empty reply twice → answer_empty;
    • tool-control text at the budget boundary → answer_tool_text;
    • a tool call written as text twice in an ordinary turn (Keep a tool call written as text out of the chat answer in every turn, and send the turn back once #946) → answer_tool_text;
    • a first request answered with 402, 429, 503 or 401 → model_out_of_credit, model_rate_limited, model_unavailable or model_rejected;
    • a save rejected by a trigger → answer_not_saved;
    • the_finalization_reasons_keep_their_codes (unit test).
  • Stream test (chat_stream_tests.rs): a producer that disappears ends in stream_ended.

  • Web (chatStream.test.ts):

    • a coded frame is worded from err.*, both on a new request and on a reattachment;
    • an unknown code keeps the server's sentence;
    • a plain-text frame is shown as it came.
  • On dev (b07af8f), I added only the tests that compile there: the handler tests, the terminal contract and the web test. Each fails:

    • the handler tests: "the error frame is not JSON";
    • the terminal contract: on its first failure row;
    • the web test: two of its three cases, because the raw JSON is shown.

    The unit test and the stream-test change use the new function and the new signature, so they were left out of that run.

  • With this change:

    • cargo fmt --all --check passes.
    • cargo clippy --workspace --all-targets -- -D warnings passes.
    • cargo test -p utopia-server api::: 174 passed.
    • cargo test --workspace with UTOPIA_TEST_REQUIRE_DB=1 and UTOPIA_TEST_REQUIRE_PDFTOTEXT=1: 1189 passed, 0 failed, 5 ignored (the existing #[ignore] tests).
    • cargo build --workspace passes.
    • pnpm install --frozen-lockfile, pnpm test (145 passed) and pnpm build pass.

Limits

  • answer_failed is the catch-all. Rarer failures share it, such as a final answer that timed out or a finish reason we do not know.
  • The English message stays in the frame for MCP, logs and clients that do not localize.

… in the reader's language

Signed-off-by: dada-yan <BinjunYann@gmail.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stream's error frame now carries the same {error, code} envelope a refused request does, so the interface words it and MCP, logs and older clients keep the English sentence. Reading the model side's codes from the error types utopia_llm already distinguishes, and pinning the finalization reasons with a test that builds them from finalization_error itself, is the right way to keep the codes honest. Tightening the terminal contract to require a code on every failure row is the part that keeps this true. Merging.

@WaylandYang
WaylandYang merged commit 5838437 into deeplethe:dev Sep 26, 2026
@Maya-Kid
Maya-Kid deleted the fix/chat-stream-error-codes branch September 26, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants