iframe-proxy

alkinun · 2026-08-16T06:25:48Z

Summary

add a truncate_oldest overflow policy that drops complete oldest turns while preserving system instructions, the active user task, and atomic tool-call/result exchanges
enable rolling context automatically for Studio local GGUF chats without deleting or hiding saved transcript history
preflight with the loaded model tokenizer and real chat template before standard generation, every tool-loop iteration, and final synthesis
surface truncation through an OpenAI-compatible metadata chunk and a one-time, active-thread Studio notice

Safety

keep llama.cpp --no-context-shift; token-level KV rotation remains disabled
never clip the latest request, and retain the normal context error when protected content cannot fit
skip proactive rolling for multimodal prompts because llama-server token counting does not include media tokens accurately
preserve strict role alternation and tool-call/result pairing after eviction

Verification

1592 relevant backend tests passed (1 skipped)
all 2721 frontend tests passed
npm run typecheck
npm run build
Ruff and git diff --check passed
completed backend, frontend, and final full-diff reviews with no release-blocking findings

Closes #7472

for more information, see https://pre-commit.ci

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1949328f57

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…feat/rolling-context-window

for more information, see https://pre-commit.ci

alkinun · 2026-08-16T06:48:08Z

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9388a75919

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…feat/rolling-context-window

alkinun · 2026-08-16T07:05:15Z

@codex review

chatgpt-codex-connector · 2026-08-16T07:09:29Z

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: a97e0ab589

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

alkinun · 2026-08-16T07:18:56Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a97e0ab589

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for more information, see https://pre-commit.ci

alkinun · 2026-08-16T08:48:00Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb4578918f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…feat/rolling-context-window

…window # Conflicts: # studio/backend/tests/test_gguf_completion_usage.py

for more information, see https://pre-commit.ci

danielhanchen · 2026-08-16T09:19:54Z

Before/after evidence for the rolling context window, from two isolated Studio installs (install.sh --local at the PR's merge base 135147814 and at head b76971675), same box, same scene.

Setup, identical on both sides: unsloth/Llama-3.2-1B-Instruct-GGUF Q4_K_M loaded at max_seq_length=2048, a 28-message thread seeded through the chat-history API so the transcript is byte-identical on both, then one short follow-up ("which topic was reference note 14 about?"). The seeded history renders to 3219 prompt tokens against a 2048-token window, so the turn overflows by construction.

BEFORE the turn dies: "Message too long: 3219 tokens exceeds the 2048-token context window. Try increasing the Context Length in Model settings, or shorten the conversation.", with a Retry button and no assistant message. AFTER it completes, with the notice and an answer that is still grounded in the kept end of the window.

Numbers read back from the same servers that were photographed:

	BEFORE	AFTER
assistant replied	no	yes
assistant text	(none)	"Reference note 14 was about tidal mills."
dropped_messages	n/a	16
prompt tokens	3219 (rejected)	3219 -> 1401
context readout	3.2k / 2.0k (red)	1.4k / 2.0k (amber)
saved messages in thread	30	30

The last row is the one worth pointing at: the saved transcript is the same length on both sides, so the eviction really is request-side only and nothing is deleted or hidden from the thread. The full conversation is still scrolled through in the AFTER screenshot.

alkinun · 2026-08-16T09:45:04Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b76971675b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for more information, see https://pre-commit.ci

alkinun · 2026-08-16T12:05:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 459558aa79

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

alkinun · 2026-08-16T14:32:58Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90a764f5f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for more information, see https://pre-commit.ci

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60faeefa24

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: efa95e08f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for more information, see https://pre-commit.ci

alkinun · 2026-08-16T19:41:32Z

@codex review

chatgpt-codex-connector · 2026-08-16T19:46:08Z

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: fe04a0daab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

danielhanchen · 2026-08-18T03:41:35Z

Before/after evidence for the rolling window, from a twelve-case campaign

Full results and images are on #9074, which is stacked on this branch: #9074 (comment)

Three findings belong to this PR rather than to the archive, and one of them is a defect I would fix here.

The window works and is invisible. In every case, this branch evicts correctly and keeps the chat answering where main would have refused. At 1.235x overshoot the boundary creeps 14, 14, 14, 16, 20, 20 across six turns; at 1.546x it moves eight separate times. The user is told none of it beyond a transient toast, and nothing survives a reload. That is the gap #9074's notice fills, and the kill-switch control in that campaign confirms the notice itself is this branch's behaviour rather than the archive's: with the archive switched off, the notice still renders, and only the "saved and searchable" wording and the counts go away.

A boundary that creeps every turn is worse than one that jumps. Compare the two sides of the same 12-turn run: eight boundary moves here against four on the stacked branch, for the same transcript. The sticky boundary plus the headroom ratio is doing real work, and the eight-move case is the shape a user experiences as the chat quietly forgetting a little more on every reply.

One real defect, in the error path. When a single message is larger than the whole window, this branch's inline red error says: Message too long: 12076 tokens exceeds the 4096-token context window. Try increasing the Context Length in Model settings, or shorten the conversation. The last clause is advice that cannot work, and this PR is what makes it unworkable: the earlier turns were already dropped, so there is no conversation left to shorten. #9074 fixes the wording in the toast but leaves the inline row saying it. It would be better fixed here, since the row is this PR's.

Measurement notes, in case they are useful for future work: boundary_messages is only written by #9074, so anything reading it against this branch alone gets nothing and will silently conclude that a thread which evicted 60 messages never compacted; dropped_messages is the field to use here, and the two must never be mixed inside one series, since that produces decreases neither build ever emitted.

danielhanchen · 2026-08-19T07:55:06Z

Confirmed there is no rolling context path in studio/backend/core/inference today, so long local GGUF chats still dead-end at the context limit, and this adds it with turn-aware eviction that keeps system messages and tool pairings intact. Backend CI Repo tests (CPU) is red here and has been intermittent lately, could you take a look and rerun.

…nslothai#8686 The contract still asserted the literal `const permissionMode = loadPermissionMode();`, which unslothai#8686 removed when it put a chat-scoped override in front of that read. This branch already carries the refactored store, so Repo tests (CPU) cannot pass here or on anything stacked above it. Ported main's replacement rather than dropping the check. The read is still the contract, and it is scoped to the deep-research setter so it cannot be satisfied by the initial-state constant, which is a different property and would keep passing if the read were deleted.

# Conflicts: # tests/studio/test_deep_research_frontend_contract.py

danielhanchen · 2026-08-19T13:06:59Z

@codex review

chatgpt-codex-connector · 2026-08-19T13:11:21Z

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 37303b6e98

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

danielhanchen · 2026-08-19T14:08:04Z

pre-commit.ci run

# Conflicts: # studio/backend/routes/inference.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bba75d9627

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-19T15:16:30Z

+        for part in content:
+            if not isinstance(part, dict):
+                continue
+            if part.get("type") in ("image_url", "input_audio", "audio", "input_image"):


Recognize input_video before rolling-context preflight

When a local GGUF request includes video_base64, _inject_video_part represents it as an input_video content part, but this media predicate does not recognize that type. Both plain and tool-loop generation therefore run rolling preflight on video prompts even though /apply-template token counting does not include the sampled video tokens; the prompt may be incorrectly certified as fitting, have history unnecessarily removed, or still fail with context_length_exceeded. Include input_video in this media check so video requests retain the intended multimodal skip.

Useful? React with 👍 / 👎.

Confirmed and fixed in e99cb75.

_inject_video_part (routes/inference.py:11923) writes {"type": "input_video", ...} into gguf_messages, and that is the same list the fit receives, so the predicate does see it. It listed image_url, input_audio, audio and input_image and not this one.

Added input_video to the check, with a comment saying where the part comes from so the next part type llama.cpp adds is easier to spot. Pinned by extending test_rolling_media_detection_covers_image_and_audio_parts; red before, green after. 223 passed across the overflow, video-simulation and tool-loop suites.

`_inject_video_part` writes llama.cpp's own `input_video` part into the same message list the fit is then handed, and the media predicate did not list it. Video prompts therefore ran the rolling preflight, which is skipped for media precisely because `/apply-template` token counting does not include the sampled video tokens. The prompt could be certified as fitting when it does not, or lose history it never needed to lose, and still fail with context_length_exceeded.

danielhanchen · 2026-08-19T15:46:12Z

@codex review

chatgpt-codex-connector · 2026-08-19T15:49:53Z

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: e99cb7566a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

danielhanchen · 2026-08-19T20:52:38Z

CI status on this branch, checked against the same jobs on main at this PR's merge base (e99dfe5a). All five failures are pre-existing and none are caused by this branch.

Three separate main-side repairs fall out of this, and belong in their own PR rather than here:

tests/studio/test_chat_autoload_failure_gate.py needs a resolvePreserveThinkingOnLoad stub next to the existing resolveToolsEnabledOnLoad one; Default preserve thinking on for Qwen3.8 #9096 added four uses of it inside the sliced region.
tests/studio/test_new_chat_context_recount.py needs a findLatestUserVideoBase64 stub; Desktop: make every drop zone take a drop again (#9036) #9056 added that call to refresh-context-usage.ts and the replayed body now throws before any count is attempted.
studio/frontend/tests/helpers/thread-sampling-world.ts drains a fixed 3 x 6 turns instead of draining to quiescence, which is short for the debounced persist path on node 22. Raising it to 12 x 24 on a clean main took the node-22 failures from 13 to 2.

Studio: add rolling context windows for local GGUF chat

1949328

alkinun requested a review from danielhanchen as a code owner August 16, 2026 06:25

[pre-commit.ci] auto fixes from pre-commit.com hooks

969e508

for more information, see https://pre-commit.ci

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/routes/inference.py Outdated

alkinun and others added 3 commits August 16, 2026 09:44

Studio: return rolling context metadata for non-stream chats

e970715

Merge remote-tracking branch 'fork/feat/rolling-context-window' into …

7f9593c

…feat/rolling-context-window

[pre-commit.ci] auto fixes from pre-commit.com hooks

9388a75

for more information, see https://pre-commit.ci

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/core/inference/context_window.py

alkinun added 2 commits August 16, 2026 09:58

Studio: keep original context when rolling fit fails

b4bee41

Merge remote-tracking branch 'fork/feat/rolling-context-window' into …

a97e0ab

…feat/rolling-context-window

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/core/inference/context_window.py

alkinun and others added 2 commits August 16, 2026 11:40

Studio: keep instruction groups independently protected

40dead2

[pre-commit.ci] auto fixes from pre-commit.com hooks

bb45789

for more information, see https://pre-commit.ci

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/core/inference/llama_cpp.py

Comment thread studio/frontend/src/features/chat/api/chat-adapter.ts Outdated

alkinun and others added 4 commits August 16, 2026 12:03

Studio: preserve rolling context metadata across retries

ffe3d9b

Merge remote-tracking branch 'fork/feat/rolling-context-window' into …

c597b53

…feat/rolling-context-window

Merge remote-tracking branch 'origin/main' into feat/rolling-context-…

579defd

…window # Conflicts: # studio/backend/tests/test_gguf_completion_usage.py

[pre-commit.ci] auto fixes from pre-commit.com hooks

b769716

for more information, see https://pre-commit.ci

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/core/inference/llama_cpp.py

alkinun and others added 2 commits August 16, 2026 12:50

Studio: count sanitized rolling context prompts

556ae70

[pre-commit.ci] auto fixes from pre-commit.com hooks

459558a

for more information, see https://pre-commit.ci

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/core/inference/llama_cpp.py

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/core/inference/llama_cpp.py

alkinun and others added 2 commits August 16, 2026 18:19

Studio: refit tool prompts after respawn

43f259e

[pre-commit.ci] auto fixes from pre-commit.com hooks

60faeef

for more information, see https://pre-commit.ci

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/routes/inference.py Outdated

Studio: retain later choice truncation metadata

efa95e0

chatgpt-codex-connector Bot reviewed Aug 16, 2026

View reviewed changes

Comment thread studio/backend/routes/inference.py Outdated

alkinun and others added 2 commits August 16, 2026 22:31

Studio: report clipping-only context truncation

d75a65f

[pre-commit.ci] auto fixes from pre-commit.com hooks

fe04a0d

for more information, see https://pre-commit.ci

danielhanchen mentioned this pull request Aug 17, 2026

Studio: keep and search the turns rolling context evicts #9074

Merged

danielhanchen added affects UI# codex-converged# important# new-feature# labels Aug 19, 2026

danielhanchen added 2 commits August 19, 2026 11:57

Merge remote-tracking branch 'origin/main' into r8961

37303b6

# Conflicts: # tests/studio/test_deep_research_frontend_contract.py

Merge remote-tracking branch 'origin/main' into r8961

bba75d9

# Conflicts: # studio/backend/routes/inference.py

chatgpt-codex-connector Bot reviewed Aug 19, 2026

View reviewed changes

danielhanchen merged commit 645ca5c into unslothai:main Aug 20, 2026
40 of 45 checks passed

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Uh oh!

Conversation

alkinun commented Aug 16, 2026

Summary

Safety

Verification

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 16, 2026

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

danielhanchen commented Aug 16, 2026

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

alkinun commented Aug 16, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 16, 2026

Uh oh!

danielhanchen commented Aug 18, 2026

Before/after evidence for the rolling window, from a twelve-case campaign