iframe-proxy

shimmyshimmer · 2026-08-17T02:37:30Z

What is going on

Tauri delivers OS file drops window-wide and suppresses the webview's own drop events. A zone wired only to onDrop therefore does nothing in the desktop app: no drag-over border, and the file is silently ignored.

Native drop routing (#8265) was only ever adopted in two places, image-dropzone.tsx and the create-project dialog. Every other file drop zone still relied on HTML5 handlers that never fire on desktop:

That is why it reads as random. The same file works when it lands on the chat, which the window-wide handler covers, and does nothing anywhere else.

The fix

A shared useNativeFileDrop hook claims the native drop for an element and hands back drag-over state plus the HTML5 handlers the web build still needs, so a caller keeps the single onFiles path it already had. Every dead zone above now uses it.

Two details worth calling out:

Documents upload by lease, not an inline read. The native reader only serves media inline, and a seed corpus can run to hundreds of MB, so the recipe seed route now accepts nativePathLease the same way the RAG upload routes already do.
The native path policy accepts video containers, so the reference pickers can register what the OS hands them.

Silent discards

Two more paths had the same symptom as the issue and are fixed alongside it:

The create-project drop zone stays registered while disabled, on purpose, so the drop does not fall through to the chat behind the dialog. It then did if (disabled) return, so the file vanished with no message. It now says the sources are still uploading.
Compare mode disabled the window-wide handler outright, so a file dropped on a compare view produced no overlay, no toast and no attachment. It now stays listening and refuses out loud, since the compare composer has no attachment queue to drain into.

Regression test

native-dropzone-coverage.test.ts walks src/ and fails if any file reads files out of a drag payload without either claiming the native drop or explicitly deferring to the window handler. It caught the reference picker while this branch was being put together, which is exactly what it is for.

Testing

npm run typecheck clean, npm test 2941 passing (5 new)
cargo check clean
pytest tests/test_data_recipe_seed.py 13 passing
Verified against a local Studio that the seed upload route now advertises nativePathLease

The native drop path itself needs a Tauri build to exercise by hand, so that part rests on the unit tests rather than a manual drag.

Tauri delivers OS file drops window-wide and suppresses the webview's own drop events, so a zone wired only to `onDrop` does nothing in the desktop app: no drag-over border, and the file is silently ignored. Native drop routing (#8265) was only ever adopted by the shared image picker and the create-project dialog. Every other file drop zone still relied on HTML5 handlers that never fire, which is why this reads as intermittent: the same file works when it lands on the chat, covered by the window-wide handler, and does nothing anywhere else. Adds `useNativeFileDrop`, which claims the native drop for an element and returns drag-over state plus the HTML5 handlers the web build still needs, then adopts it in the zones that were dead: - Projects -> Sources, which also had no drag-over styling at all - Data Recipes unstructured seed - Diffusion training images - Video and audio reference pickers Documents upload by lease rather than an inline read, since the native reader only serves media inline, so the recipe seed route now accepts `nativePathLease` the way the RAG upload routes already do. The native path policy accepts video containers so the reference picker can register what it is given. Also stops two silent discards with the same symptom: a claimed zone that refused a drop while disabled, and compare mode disabling the window-wide handler outright. Both now say what happened. `native-dropzone-coverage.test.ts` walks src/ and fails if a zone reads files from a drag payload without either claiming the native drop or explicitly deferring to the window handler.

…eading Two things the first pass got wrong. Keeping the window-wide listener on outside single chat also handed it model drops, so a GGUF dropped on a compare or project view would load and replace the active model. Nothing happened there before, so that is not a change this should be making. The refusal now covers every kind the handler would act on, models included. The recipe seed route also moved its extension check after the read, so a rejected 500 MB upload was pulled into memory first. Back to validating the filename before reading a byte, as it was.

64 MB sat under the reference picker's own 72 MB, so a clip the picker accepts was refused on drop. The cap is a backstop; callers keep theirs.

… read Two review findings, both correct. The diffusion dataset zone accepts .bmp, .m4v, .caption and .jsonl, which the chat attachment policy rejects outright, and .txt, which registers but cannot be read inline. Captions beside images are the documented workflow there, so wiring that zone to the attachment path would have uploaded the images and silently lost the captions. It needs its own registration and upload policy, which is more than this belongs to, so it goes back to the picker it had. The recipe seed route also read a dropped path in full before checking any limit, so a multi-gigabyte local file went into backend memory before the 413. It now refuses on the stat and bounds the read by what the block has left, in case the file grows in between.

shimmyshimmer · 2026-08-17T03:37:20Z

Went through all four. Three were right, one was already fixed.

Diffusion dataset zone (correct, and worse than described). I checked the accept list against the native policy:

diffusion accepts : .png .jpg .jpeg .webp .bmp .mp4 .mov .mkv .webm .m4v .avi .txt .caption .jsonl
rejected outright : .bmp .m4v .caption .jsonl
registers, cannot be read inline : .txt

.txt is the caption workflow the field's own help text describes (cat.png and cat.txt), so dropping images with captions would have uploaded the images and silently lost the captions. That zone needs its own registration policy and a lease-capable upload route, which is a bigger change than this belongs to. Reverted to the picker it had in 73dd714, so it is back to the honest pre-existing state on desktop.

Native read before the size check (correct). A dropped path is a local file of any size, and it was read in full before any limit. Now refused on its stat, with the read bounded by what the block has left in case the file grows in between. Test added that asserts the file is never opened when the stat already exceeds the cap; it fails against the old ordering.

Video cap versus the reference picker (already fixed). This landed on 02a18b0; 8bec354 raised MAX_NATIVE_VIDEO_BYTES to 96 MB, above the picker's 72 MB raw limit, with callers keeping their own tighter caps.

Reference picker accepting fewer formats than its input (correct, keeping it). The input is ${kind}/* and readReferenceFile takes any matching MIME, so .mpeg or .aac can be picked but not dropped. The native policy is an extension allowlist and widening it to everything ffmpeg reads is not a bounded change. Before this PR that drop did nothing at all; now it names the formats it takes. A subset with a clear message beats silence, so I would rather leave the asymmetry than grow the allowlist here.

for more information, see https://pre-commit.ci

…o the raw limit A disabled seed zone carried pointer-events-none, which takes it out of elementFromPoint, so nativeDropTargetAt could not find the target it had just registered. The disabled message was unreachable and the drop fell through to the window handler instead. MAX_NATIVE_VIDEO_BYTES was set to the reference picker's 96 MiB, but that cap bounds the data URL, not the file: the picker's own raw limit is 75497280 bytes. Rust was reading and base64-encoding up to 96 MiB, 128 MiB across the bridge, for clips the picker then rejected.

shimmyshimmer · 2026-08-17T07:16:41Z

Three new ones. Two were right, one has a fix that would make it worse.

Disabled seed zone not hit-testable (correct, and self-inflicted). elementFromPoint skips pointer-events-none, and nativeDropTargetAt walks up from whatever it returns, so disabling the zone that way un-registers it in practice. The disabledReason I passed to the hook was unreachable by construction. Now only the interactive behaviour is disabled; the element stays in the hit test. Fixed in 06c90fd, with a check in the dropzone coverage test that fails against the old class list.

Native video cap versus the raw reference limit (correct, and my earlier fix overshot). The 96 MiB figure bounds the data URL, not the file. rawLimitFor in reference-budget.ts computes the actual raw ceiling:

(96 MiB - 256 header) * 3/4, floored to a multiple of 3 = 75497280 bytes

So Rust was reading and base64-encoding up to 96 MiB, 128 MiB across the bridge, for clips the picker then rejected against 72 MiB. Set to 75497280 with a test that pins it to MAX_REFERENCE_BYTES.video rather than to a literal, so the two cannot drift again.

Queued batch tokens expiring (correct premise, wrong remedy). The loop is sequential and TOKEN_TTL is absolute from registration, both as described. But the suggestion to redeem the queued tokens up front makes it strictly worse:

native_intents.rs      TOKEN_TTL = 15 min
native_backend_lease.rs LEASE_TTL =  2 min

Minting the leases at drop time is what the previous revision did, and it fails 7.5x sooner. Redeeming a token does not extend it either, since entry_for_operation does not touch expires_at_ms. That leaves either weakening the token lifetime or running the uploads concurrently, and concurrency races the per-block total-size check the route does on each request. A later file in a batch behind a 15 minute extraction fails with a clear per-file message and can be re-dropped, which I would rather keep than trade for either of those. Leaving it.

shimmyshimmer · 2026-08-17T08:01:57Z

Same class as the seed queue above, and the mechanism is right: the loop in use-rag-documents.ts is sequential and TOKEN_TTL runs from registration, not from redemption. But the exposure is materially smaller here than on the path it is being compared to, and the suggested remedy is still the one that makes it worse.

These two paths are not equivalent. The seed route extracts synchronously inside the POST (_extract_text_from_file, seed.py:545), so a slow document holds the loop for as long as extraction takes. The RAG route does not: start_ingestion returns a job_id and indexing runs asynchronously, which is why the hook polls on hasIndexing. What the POST actually does is _save_native_path_upload, a bounded local copy capped at MAX_UPLOAD_BYTES (200 MB per file by default). Reaching 15 minutes means copying tens of GB of local files, not waiting on one slow parse.

Preserving the tokens up front is the worse trade, on both paths:

native_intents.rs       TOKEN_TTL = 15 min
native_backend_lease.rs LEASE_TTL =  2 min

Redeeming early converts a 15 minute budget into a 2 minute one. That is what the first revision of this PR did, and it is why the current code mints per upload. Redeeming does not extend the token either, since entry_for_operation never touches expires_at_ms.

What a real fix needs. Either a sliding expiry (new IPC surface, and it makes an idle token's lifetime effectively unbounded), or concurrent uploads, which race the per-block total-size check the seed route performs on every request. Neither belongs in a PR whose subject is making these zones accept a drop at all.

For what it is worth, the failure is a clear per-file message on a row the user can re-drop, on a path that before this PR ignored the drop entirely. I would rather ship the bounded limitation than take the 7.5x regression or the security-relevant rewrite. Leaving both instances as they are.

shimmyshimmer · 2026-08-18T09:04:00Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 06c90fd3c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-18T09:07:10Z

+    _require_within_budget(size_bytes, budget)
+
+    try:
+        with grant.canonical_path.open("rb") as source:


Bind the opened file handle to the verified grant

When the dropped path is replaced between verify_native_path_lease and this open—whether with a symlink or an atomically renamed regular file—Path.open follows and reads the replacement without comparing the opened handle's identity to the signed grant. This defeats the lease fingerprint and can extract a different file than the user dropped; open without following links and validate the handle's size, timestamps, device, and inode before reading.

Useful? React with 👍 / 👎.

The path is never webview-named: Rust signs what the OS handed it and verify_native_path_lease re-fingerprints it. Reaching this needs a local attacker racing the filesystem on the user's own machine, which is not legitimate real-session input.

chatgpt-codex-connector · 2026-08-18T09:07:10Z

+
+    try:
+        with grant.canonical_path.open("rb") as source:
+            content = source.read(budget + 1)


Bound the native read by the per-file cap

When a file grows after the preceding stat, this read is bounded only by the block's remaining budget, which can be nearly 1 GiB even though the per-file limit is 500 MiB. Fresh evidence in this revision is the new source.read(budget + 1) call: it can allocate and read roughly twice the allowed file size before _require_within_budget rejects it, potentially stalling or exhausting the backend; use one byte beyond the tighter of the remaining block budget and per-file cap.

Useful? React with 👍 / 👎.

The 500 MB per-file cap is already enforced on the stat() before open(), so read(budget+1) can only exceed it if the file grows mid-read, and it is bounded at about 1 GiB rather than unbounded.

danielhanchen · 2026-08-18T12:01:39Z

Review summary

Verdict: useful. Real issue, and a genuine PR-caused CI break fixed in a3cb8e1.

Before / after

Before, several desktop drop zones did not accept a native drag: the disabled seed zone was pointer-events-none, so a real file drag on it was silently swallowed with no feedback at all, and the native video cap did not match the reference picker's own limit. After, a disabled zone stays hit-testable and answers with the busy toast, the native video cap matches the picker, and the seed route checks the size budget before opening the file rather than after reading it.

The CI break was this PR's

Unsloth Tauri CI is green on main but red here on Linux, macOS and Windows. Commit 8bec354b9 changed the error string to "Only chat image, audio and video attachments can be read inline." and left the assertion at native_intents.rs:820 matching the old "Only chat image and audio attachments". The comma after image makes the assertion unsatisfiable, so document_token_is_refused_by_the_image_reader panics.

Verified on a clean tree at head: 370 passed; 1 failed. With the one-line fix: 371 passed; 0 failed. Sibling PR #9057 carried the identical defect and has the same fix.

Separately, pre-commit.ci - pr is red as "error during mergeable check" because mergeable_state is dirty - two mechanical conflicts with main, in studio/backend/tests/test_data_recipe_seed.py (vs #8962) and studio/frontend/src/features/rag/components/project-sources-panel.tsx (vs #8756).

On the 10 Codex items

Six genuine, four rejected. Five of the six were already fixed on the branch; each thread names the commit. Two rejections are worth the detail:

The project-sources-panel item looks mechanically identical to the seed TOCTOU one, but the timing is not: ingestion.start_ingestion spawns a thread and returns immediately, so the per-file blocking work is a 200 MB-capped streaming copy plus sha256, about 1-2s. Exhausting a 15-minute TTL there would need order-of-100 GB.
The two seed.py path items need a local attacker racing the filesystem on the user's own machine. Rust signs what the OS handed it and verify_native_path_lease re-fingerprints it, so the path is never webview-named. On the read bound specifically, the 500 MB per-file cap is already enforced on the stat() before open(), so read(budget+1) only exceeds it if the file grows mid-read, and it is bounded at about 1 GiB rather than unbounded.

I also independently recomputed the video cap rather than trusting the number: rawLimitFor(96*1024*1024) gives floor(((100663296-256)*3/4)/3)*3 = 75497280, an exact match for MAX_NATIVE_VIDEO_BYTES. And no chat regression, since classifyDropPaths still has no video arm, so a dropped .mp4 on the chat window remains unsupported.

One genuine item deliberately not fixed here

The native batch TTL item is real, and I measured it rather than agreeing on principle. The TTL is stamped once in insert_entry and never renewed, while seed.py:545 runs pymupdf4llm.to_markdown synchronously inside the request the loop awaits. Measured on real arXiv PDFs at about 3.24 s/MB (326 pages / 24.67 MB in 80s), so roughly 278 MB of ordinary PDF exhausts the 15-minute TTL - against a zone that advertises 500 MB per file and 1 GB per block. The tail of a legitimate batch is pruned and fails with "Native path token is unavailable or expired".

It is not fixed in this PR because the available fix is 220 lines of Rust that change token lifetime semantics, which does not belong in a drop-zone PR. The narrower alternative is to make seed extraction asynchronous the way RAG ingestion already is, which removes the exposure without touching the token model. Going out separately either way.

Simulation

The browser matrix genuinely applies here, so it was driven rather than asserted: 33 scenarios across Chromium, Firefox and WebKit, plus engine-level CDP drags. Edge is Chromium, so that is three engines, not four. The disabled-zone fix was reproduced at engine level in all three - with the pre-PR markup a real file drag is silently swallowed and elementFromPoint returns #root; with the PR markup it produces the busy toast. Click suppression is unaffected, since handleClick guards on disabled independently.

Rust cargo test 377 passed after the fix, npm test 2946 passed, tsc clean.

Backwards compatibility: useNativeDropTarget no-ops outside Tauri, so the plain-browser HTML5 handlers remain live, and multipart upload is unchanged when no nativePathLease is sent - an old frontend against a new backend is fine, and an older desktop build that never mints grant tokens does not lose drag-drop.

Below the bar, noted

toastNothingAccepted's folder heuristic mislabels extension-less files like README, and the new coverage test's regex only matches the template-literal form of pointer-events-none, so a cn(...)-style reintroduction would slip past it.

chatgpt-codex-connector · 2026-08-18T13:19:01Z

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: a3cb8e1c0e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

danielhanchen · 2026-08-18T13:19:15Z

UI evidence

BEFORE is the merge base def58ea6358213f59398704d70f99b45c892db06, AFTER is the head of this branch a3cb8e1c0e73c41511c5dfe8dec837bea5f06770. Two separate install.sh --local builds, one per commit, driven by the same script.

Read this first: the regression itself is not in the picture. #9036 is a Tauri problem. The desktop webview never sees its own drop events, so a zone wired only to onDrop is dead in the app. Both sides below are web builds in headless Chromium, where isTauri is false, so every native branch in useNativeFileDrop is on the unused side. Showing the actual desktop fix needs a packaged app plus a real OS drag, which is not reachable headless on this host.

What is web-reachable is the rewrite the fix required. The project Sources panel used to carry a bare onDragOver={e => e.preventDefault()} and an onDrop that shipped whatever arrived straight to the upload. It had no drag state, so a drag over it drew nothing, and no accept filter, so nothing was refused locally. Routing it through the new hook gives it both on the web path as well.

Both drags are synthesised with a real DataTransfer carrying a real File, so dataTransfer.types contains Files and the hook's own file-drag guard is exercised rather than bypassed.

Facts that moved, read off the same servers that were photographed:

fact	BEFORE	AFTER
`dragover_highlight`	`false`	`true`
`zone_class_dragover`	identical to its idle class	gains `border-primary/60 bg-primary/5`
answer to a dropped `.zip`	`Couldn't upload corpus.zip` / `Unsupported file type '.zip'. Allowed: ['.docx', '.htm', '.html', '.markdown', '.md', '.pdf', '.txt']`, from the server	`That file type can't be dropped here` / `Accepts .pdf,.txt,.md,.markdown,.docx,.html,.htm.`, from the browser
`/api/rag/projects/<id>/documents` after that drop	`{"documents": []}`	`{"documents": []}`

The rest of the panel is unchanged between the halves, which is what makes the border the only thing that moved in shot 1.

What this pair does not cover:

the desktop path, which is the actual bug: no Tauri window, no OS drop, no native path lease, no registerNativeAttachmentPath call.
the other zones this PR touches. The Compare-view refusal in chat-page.tsx, UnstructuredDropZone staying hit-testable while disabled, and ReferenceMediaPicker are all untouched by this scene.
the "busy zone" refusals. Nothing here was uploading, so disabledReason never fired on either side.
the backend half (data_recipe/seed.py) and native_intents.rs.

Useful, with the caveat stated: it demonstrates the web-side behaviour change in one of the rewritten zones, and it does not and cannot demonstrate the desktop drop this PR exists to fix.

Imagineer99 · 2026-08-18T14:28:42Z

Tested on Windows: dropping into an existing chat works, but it fails in a fresh empty chat even with a model loaded (“Couldn’t start a chat for these documents”). The file is detected, but the new chat isn’t created.

# Conflicts: # studio/backend/tests/test_data_recipe_seed.py # studio/frontend/src/features/rag/components/project-sources-panel.tsx

for more information, see https://pre-commit.ci

danielhanchen · 2026-08-18T17:10:59Z

Merged current main in (ed8edc0a7, now under pre-commit.ci's follow-up) rather than rebasing, so nobody's local copy breaks. This PR was CONFLICTING / DIRTY, which is why the only red check was pre-commit.ci - pr: its result page says "merge conflict" at the mergeable-check stage, so no hook ever ran and GitHub could not compute a merge ref for the other workflows either.

Two conflicts, both resolved in favour of keeping both sides:

project-sources-panel.tsx. This PR generalised handleFiles into handleItems(items: RagUploadItem[]) for the desktop-drop path; main changed the post-upload half from a second invalidateProjectSources to announceProjectSourcesUpdated, with a comment explaining that announcing before the upload would refetch and resurrect a row the panel had already dropped. I kept your signature and took main's announce semantics, since the two changes are orthogonal and main's is the newer contract.
test_data_recipe_seed.py. Purely additive on both sides: your two native-drop size-cap tests and main's _BlockPlugin plus plugin-resolution tests. Both kept.

After the merge: backend seed tests 18 passed 1 skipped, cargo test in studio/src-tauri 373 passed 0 failed, frontend typecheck clean, frontend suite 3785 passed with one failure, tests/queued-model-capabilities.test.ts, which is red on main today for an unrelated extensionless import that #9192 fixes.

danielhanchen · 2026-08-18T18:54:51Z

Codex approved this at a3cb8e1 and the head is two commits ahead, so the convergence check reports the approval as sitting on an ancestor. Those two commits are a merge of main and a pre-commit.ci formatting pass, not changes to this PR.

Diffing the PR's own content against its merge base at both revisions: the approved diff and the head diff are identical apart from hunk line offsets and two blank lines around a test class that the merge shifted. Nothing this PR changes has moved since the approval, so I am treating it as approved at head rather than spending another review round on it.

danielhanchen · 2026-08-19T07:52:53Z

Could you get Backend CI (Python 3.13, Repo tests CPU) and the two Frontend CI jobs green, since those pass on main?

…slothai/unsloth into r9056

…9057) * Studio: attach video to a chat, and say why when it is unavailable llama.cpp takes video through its OpenAI-compatible chat endpoint as an `input_video` content part, but only when the mmproj declares video, the binary was built with video support and ffmpeg is installed. It reports that verdict at /props under modalities.video. Nothing about the GGUF alone can tell us, so that is what Studio now reads. Frontend: video joins the drop classifier and its own pending queue beside images and audio, and a VideoAttachmentAdapter takes one clip per message from the picker or a drop. When the model cannot take video the adapter names all three possible causes instead of letting llama-server refuse the request later. Backend: video_base64 on the chat request is forwarded whole as an input_video part, since llama-server owns the frame sampling and there is nothing useful to transcode. has_video_input rides the same path as has_audio_input out to the model row. Compare mode is left out on purpose: video_base64 targets the single loaded GGUF, so at most one side could answer. Dropping a clip there now says so rather than ignoring the file. Size caps line up across the three hops (64 MB in the desktop reader, in the composer and in the route) so no hop accepts what the next refuses. * Studio: carry the video capability through, and cover the passthrough path Three review findings, all correct. syncModelCapabilities took has_video_input but never copied it into the row, and /api/models/list omits it for the active GGUF, so the adapter read false after every load and refused video even when /props reported it. The feature did not work in its main path. The tool and response_format passthrough returns before the injection and forwards an explicit field list, so a clip rode along nowhere and the model answered without it. Refused now, the way audio already is there. The size cap floored the base64 inflation, so a clip of exactly the size the composer allows was refused with a 413, and the data URI header was counted against the payload. Padded ceiling, measured after stripping. * Studio: carry the video capability through every hop, and refuse it where it cannot be served Three separate places map backend capability flags onto a model row and each one dropped the video flag: the direct status adoption, the queued-run capability Pick, and (fixed earlier) syncModelCapabilities. The adapter reads that row, so any of them leaves video refused on a model that supports it. Covered by a rule rather than three spot checks. Injection lives in the GGUF branch, so an external provider or a local transformers model answered as if no clip were attached. Both now refuse, as does token counting, which cannot inject the frames it would need to count. The size check also moved ahead of the automatic model switch so an oversized clip does not evict a working model before the 413, and video now votes in the pre-switch projector requirement alongside audio. The video drain's read-failure toast said 'audio', inherited from the audio drain it was cloned from. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: name the attached modality in the pre-switch refusal Adding video to require_vision made the shared rejection reachable for a request carrying only a clip, but its text is fixed at 'image or audio input', so the user who attached a video was told about modalities the request never carried. The label now follows what is attached, and defaults to the existing wording so the image-only callers are unchanged. * Match the document-refusal test to the message it now returns * Send the API key on the /props readback and skip video in the context recount The /props probe went out without an Authorization header, so under UNSLOTH_DIRECT_STREAM=1 llama-server answered 401 and video capability never came back. Context recount already bails on images and audio because toOpenAIMessages has no branch for them; video has the same property and was missing the bail, so the usage bar priced a text-only prompt and stringified megabytes of base64 on the UI thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Send a picked clip under the container its extension names The accept list carries extensions as well as mime types because the browser's answer is unreliable for mkv and some mov files, and the picker takes those files on the extension. Only an empty type was being replaced, so a clip the browser called application/octet-stream kept that type into the attachment, and the request builder recognises a file part only when its mimeType matches ^video/. The clip was attached, sent and dropped, and the model answered as though nothing were there, which is the silent drop this PR exists to remove. The table mirrors the one in native_intents.rs, so a clip read by the desktop reader and one picked in the browser reach the route the same way. --------- Co-authored-by: shimmyshimmer <182633334+shimmyshimmer@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bf2e3bb9ef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-19T12:14:28Z

    block_dir = UNSTRUCTURED_UPLOAD_ROOT / block_id
+    # Reads 0 for a block with no directory yet, so this does not create one for
+    # an upload that is about to be refused.
+    budget = UNSTRUCTURED_RECIPE_UPLOAD_TOTAL_MAX_BYTES - _get_block_total_size(block_dir)


Recheck the block quota after the awaited upload read

When two large multipart uploads target the same block concurrently, both can calculate the same remaining budget here and then yield in await file.read(). Each request subsequently validates against that stale snapshot and writes synchronously, allowing their combined size to exceed UNSTRUCTURED_RECIPE_UPLOAD_TOTAL_MAX_BYTES. Previously the total was measured after the awaited read; recompute/reserve the quota after that await (with serialization if multiple workers can write the block) before saving.

Useful? React with 👍 / 👎.

Not reachable here. The drop zone uploads strictly one at a time (unstructured-drop-zone.tsx awaits uploadUnstructuredFile inside its for loop), and a block's dialog is a single modal in a single-user local app, so two multipart uploads never target the same block_id concurrently in a real session. Reaching this needs two clients dropping into the same block at the same instant, and the outcome is a soft total-size cap overshooting, not a crash or lost data. Worth noting the pre-existing code had the same shape: the total was measured and the file written in one synchronous stretch, but nothing serialised two requests either.

danielhanchen · 2026-08-19T12:43:43Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 104ea3c40c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-19T12:48:33Z

+        # Video rides that projector too. Its own /props gate can only run after
+        # the load, so this at least keeps a text-only target from evicting a
+        # working model to serve a clip it could never take.
+        _needs_vision = _needs_image or bool(payload.audio_base64) or bool(payload.video_base64)


Treat video as requiring an image-capable projector

When automatic model switching is enabled and a video request names an audio-only GGUF, this sets require_vision but leaves _needs_image false. _target_is_vision(..., require_image=False) deliberately accepts any companion projector, including an audio-only one, so Studio evicts the currently working model, loads the incompatible target, and only then rejects it when /props reports no video support. Include video in the image-capability requirement so audio-only projectors are rejected before the swap.

Useful? React with 👍 / 👎.

This needs auto-switch turned on (DEFAULT_OPENAI_AUTO_SWITCH_ENABLED is False) plus a video request that names an audio-only GGUF, a target that cannot serve the clip either way: the request 400s identically, only after a swap instead of before it. The pre-swap gate is there to stop a text-only target evicting a working model, and that case still resolves to no mmproj and is rejected before the load.

for more information, see https://pre-commit.ci

…i#9342) * Stub the two helpers the sliced harnesses started importing Backend CI's "Repo tests (CPU)" has been red on main since 588405d, with 111 failures in two files. Both are the same defect, and neither is about what the tests claim to be testing. These harnesses replay real studio source with its import block stripped, so every name the sliced region calls has to be defined in the harness. Two landed without one: refresh-context-usage.ts findLatestUserVideoBase64 (unslothai#9056, decline to price a prompt carrying video) chat-runtime-store.ts resolvePreserveThinkingOnLoad Both product changes are right. The first is the more dangerous shape: with no stub the replayed body throws a ReferenceError, the effect bails, `counts` stays 0, and the assertion reads "the empty New Chat view must be priced exactly once" -- a pricing bug that does not exist. 41 tests failed that way, and bisecting main was the only way to see it, because the message points at the wrong thing. test_chat_autoload_failure_gate.py already guards this and said exactly what was missing, which is why its 70 failures named resolvePreserveThinkingOnLoad outright. test_new_chat_context_recount.py had no such guard; it does now, built the same way -- parse the real import list, assert the harness defines each name. Mutation-tested: dropping the video stub turns it red with the name in the message. The preserve-thinking stub is the real resolver's rule verbatim, since no scenario here sets a stored preference. tests/studio: 0 failures, from 111. * Count nesting when finding install.sh's GPU-detection fallback With the harness stubs in place, Repo tests (CPU) is down to one failure, and it is the last of the same kind: correct product code reported as a defect by a scan that reads the file too loosely. test_no_torch_backend_auto_outside_fallback allows --torch-backend=auto only inside install.sh's "GPU detection failed" branch, and located the end of that branch as the first line equal to `fi` after the comment. unslothai#8670 put a `case` into the branch to choose the desktop install spec, with an `if`/`fi` inside one of its arms. That `fi` is now the first one, so the block ended four lines early and the branch's own install call -- line 5280, the one the test exists to permit -- was reported as a primary path using the flag. The end is now found by counting nesting. `case` counts too: it closes with `esac`, so an `if`/`fi` inside a case arm would still throw off a counter that only tracked `if`. Added test_the_fallback_range_reaches_the_end_of_the_branch, because the bug here was in the range and not in what the range was used for, so nothing was checking the range itself. It pins both ends: the block contains the branch's first `if` and both of its --torch-backend=auto calls, and does not run past its closing `fi`. Mutation-tested both ways. Adding --torch-backend=auto outside the branch still fails the original assertion; reverting the nesting count to the first-`fi` behaviour fails the new one. tests/python/test_cross_platform_parity.py 78 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

… leg (unslothai#9348) * Stub the two helpers the sliced harnesses started importing Backend CI's "Repo tests (CPU)" has been red on main since 588405d, with 111 failures in two files. Both are the same defect, and neither is about what the tests claim to be testing. These harnesses replay real studio source with its import block stripped, so every name the sliced region calls has to be defined in the harness. Two landed without one: refresh-context-usage.ts findLatestUserVideoBase64 (unslothai#9056, decline to price a prompt carrying video) chat-runtime-store.ts resolvePreserveThinkingOnLoad Both product changes are right. The first is the more dangerous shape: with no stub the replayed body throws a ReferenceError, the effect bails, `counts` stays 0, and the assertion reads "the empty New Chat view must be priced exactly once" -- a pricing bug that does not exist. 41 tests failed that way, and bisecting main was the only way to see it, because the message points at the wrong thing. test_chat_autoload_failure_gate.py already guards this and said exactly what was missing, which is why its 70 failures named resolvePreserveThinkingOnLoad outright. test_new_chat_context_recount.py had no such guard; it does now, built the same way -- parse the real import list, assert the harness defines each name. Mutation-tested: dropping the video stub turns it red with the name in the message. The preserve-thinking stub is the real resolver's rule verbatim, since no scenario here sets a stored preference. tests/studio: 0 failures, from 111. * Let the duplicate-load fast path read the binary it now has to compare Backend CI's "(Python 3.13)" leg has been red on main since e99dfe5, "Studio: use a custom llama.cpp build (unslothai#9292)", with three failures. All three are the tests, not the change. Two in test_metal_paravirtual_guard.py forbade BOTH _kill_process and _find_llama_server_binary on a duplicate /load, under one message: "tore down a healthy server on a duplicate /load". Only the first is teardown. unslothai#9292 put _binary_changed_since_launch() on the fast path, which reads the current llama-server to see whether a different one is installed than the live process was launched from -- exactly what a custom build has to check, since an identical request must still reload when the binary underneath it changed. So the read is correct and forbidding it reported a lookup as a teardown. _kill_process stays forbidden; the binary lookup now returns a fixed path, so the revision comparison sees no change and the fast path is still what is exercised. One in test_vram_budget_settings.py asserted the compacted source of _serial_load_scope ENDS WITH self._vram_fraction_pending=None. unslothai#9292 added a sibling clear, _binary_revision_pending, so the release is still in the finally but no longer last in it. The assertion now checks it is inside the finally, which is the property the test names. Mutation-tested: deleting the vram release from the finally still turns it red. Note on the leg name: 3.11 is not a second copy of this suite. The matrix is 3.13 scope=full and 3.11 scope=floor-spot-check, about 40 tests in three files, so "3.11 green, 3.13 red" is not a version difference -- 3.13 is simply the only leg that runs these. Reproduced locally on 3.13.12. test_metal_paravirtual_guard.py 119 passed, test_vram_budget_settings.py 90 passed. Test files only; no product code touched.

shimmyshimmer mentioned this pull request Aug 17, 2026

Studio: attach video to a chat, and say why when it is unavailable #9057

Merged

This comment was marked as resolved.

Sign in to view

shimmyshimmer added 2 commits August 16, 2026 20:13

Desktop: size the native video cap to the largest client-side limit

8bec354

64 MB sat under the reference picker's own 72 MB, so a clip the picker accepts was refused on drop. The cap is a backstop; callers keep theirs.

shimmyshimmer requested a review from danielhanchen as a code owner August 17, 2026 03:34

This comment was marked as resolved.

Sign in to view

pre-commit-ci Bot and others added 2 commits August 17, 2026 03:39

[pre-commit.ci] auto fixes from pre-commit.com hooks

b48bf95

for more information, see https://pre-commit.ci

This comment was marked as resolved.

Sign in to view

chatgpt-codex-connector Bot reviewed Aug 18, 2026

View reviewed changes

Match the document-refusal test to the message it now returns

a3cb8e1

unslothai deleted a comment from chatgpt-codex-connector Bot Aug 18, 2026

danielhanchen mentioned this pull request Aug 18, 2026

Studio: bring the Llasa top_p default inside the range the API accepts #9194

Merged

danielhanchen and others added 2 commits August 18, 2026 17:07

Merge remote-tracking branch 'origin/main' into r9056

ed8edc0

# Conflicts: # studio/backend/tests/test_data_recipe_seed.py # studio/frontend/src/features/rag/components/project-sources-panel.tsx

[pre-commit.ci] auto fixes from pre-commit.com hooks

dae3a0e

for more information, see https://pre-commit.ci

unslothai deleted a comment from chatgpt-codex-connector Bot Aug 18, 2026

danielhanchen added affects UI# important# labels Aug 19, 2026

danielhanchen and others added 3 commits August 19, 2026 12:04

Merge remote-tracking branch 'origin/main' into r9056

61bc640

Merge branch 'studio/desktop-drag-drop-9036' of https://github.com/un…

65c5429

…slothai/unsloth into r9056

chatgpt-codex-connector Bot reviewed Aug 19, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into r9056

104ea3c

chatgpt-codex-connector Bot reviewed Aug 19, 2026

View reviewed changes

[pre-commit.ci] auto fixes from pre-commit.com hooks

06d5d99

for more information, see https://pre-commit.ci

danielhanchen merged commit 588405d into main Aug 19, 2026
42 of 48 checks passed

danielhanchen deleted the studio/desktop-drag-drop-9036 branch August 19, 2026 13:10

This was referenced Aug 19, 2026

Studio: add rolling context windows for local GGUF chat #8961

Merged

Stub the two helpers the sliced harnesses started importing #9342

Merged

Get Repo tests (CPU) back to green: 113 failures from three test-harness gaps #9344

Closed

Surface	Before
Projects -> Sources (the markdown case in the issue)	dead, and no drag-over styling at all
Data Recipes unstructured seed	dead
Diffusion training images	explicitly `if (isTauri) return`
Video and audio reference pickers	dead

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Uh oh!

Conversation

shimmyshimmer commented Aug 17, 2026

What is going on

The fix

Silent discards

Regression test

Testing

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

shimmyshimmer commented Aug 17, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

shimmyshimmer commented Aug 17, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

shimmyshimmer commented Aug 17, 2026

Uh oh!

shimmyshimmer commented Aug 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen commented Aug 18, 2026

Review summary

Before / after

The CI break was this PR's

On the 10 Codex items

One genuine item deliberately not fixed here

Simulation

Below the bar, noted

Uh oh!

chatgpt-codex-connector Bot commented Aug 18, 2026

Uh oh!

danielhanchen commented Aug 18, 2026

UI evidence

Uh oh!

Imagineer99 commented Aug 18, 2026

Uh oh!

danielhanchen commented Aug 18, 2026

Uh oh!

danielhanchen commented Aug 18, 2026

Uh oh!

danielhanchen commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen commented Aug 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

danielhanchen commented Aug 19, 2026 •

edited

Loading