Repository navigation
Early responses to large uploads hang when the relay forwards a cloned request body #737
Description
Early upload rejection can stall
No credentials, account identifiers, project references, private content or actual photos are included.
Problem
An Edge Function returning an early error for a finite 2,097,153-byte upload can leave the HTTP client waiting until its own timeout. This prevents a bounded photo endpoint from delivering its HTTP 413 response. A minimal local function with no Auth, database, Storage or Flare code reproduces the issue. Even an immediate response without reading or explicitly cancelling the body can stall.
Environment and observations
- Windows host with Docker Desktop; Supabase CLI 2.110.0.
- CLI reports supabase-edge-runtime-1.74.2, compatible with Deno 2.1.4.
- Normal CLI-generated relay, with no runtime patches, used for the failing tests.
- Finite 2 MiB + 1 body: immediate return, awaited cancellation, unawaited cancellation and a streamed error response timed out after 5 seconds in the minimal reproduction.
Connection: closeon the response did not resolve it. - Fully consuming that same finite body before returning produced HTTP 413 in approximately 60 ms. This is a diagnostic control, not a proposed production mitigation: unrestricted draining would defeat the resource bound.
- A bounded-drain control returned 413 for 2 MiB + 1 in 77 ms but a finite 4 MiB request timed out after 5 seconds when that drain budget was exceeded. This just moves the failure to a larger request.
- The real authenticated photo endpoint independently times out after 20 seconds for both fixed-length and chunked 2 MiB + 1 requests. Exact 2 MiB reaches JPEG validation; ordinary photo upload and removal work.
- Earlier hosted Beta checks showed the same two authenticated boundary timeouts. We cannot inspect the managed relay and do not claim its implementation has been proved identical to the local cause.
Local cause and counterfactual
The CLI-generated relay forwards new Request(t,e.clone()), leaving the original request branch unused. In Edge Runtime v1.74.2, UserWorker.fetch pipes the forwarded body to the user worker and waits for both the body-pipe promise and the response promise before releasing the response:
Exact upstream implementation.
When the worker stops reading early, piping tries to cancel its source. Cancellation of the cloned branch can remain pending while the other branch is unconsumed. A standalone Web Streams model reproduces that pending wait and releases it when the unused branch is cancelled.
An earlier isolated diagnostic removed request cloning only in the generated local relay; all 26 endpoint checks then passed with unchanged application handlers. That diagnostic change was restored and was never deployed. Direct Deno with the same application handlers also passed those checks. These controls isolate the local forwarding/cancellation interaction; they do not establish a supported hosted workaround.
Minimal reproduction
The index.ts included below is the exact local diagnostic function. It uses no credentials or database. Its drain modes exist only as controls and must not be deployed as a public function.
- In a disposable local Supabase project, put it at
supabase/functions/photo-body-diagnostic/index.tsand configure[functions.photo-body-diagnostic]withverify_jwt = false. - Use local API port 58121, start the local stack and run
supabase functions serve. The probe refuses bodies larger than 5 MiB and targets only127.0.0.1:58121. - Wait until
OPTIONS /functions/v1/photo-body-diagnosticreturns 204. Do not edit watched function files during checks, because the CLI restarts the runtime. - Run
node transport-probe.mjs return cancel cancel-nowait drain response-stream. Default body size is exactly 2 MiB + 1. - The early-response cases should promptly return 413; they instead time out. The finite full-drain control returns 413.
- The
bounded-drainmode also demonstrates that limiting discard work moves the same failure to larger bodies. SetPHOTO_PROBE_BYTES=4194304for that finite 4 MiB control.
Assistance requested
Please confirm whether the managed Edge Functions gateway has the same forwarding/cancellation issue, and provide either a provider fix or a supported way to deliver an early 413 while preserving strict byte/time bounds on rejected input. Increasing the accepted photo limit, unrestricted body draining, editing generated local runtime files, and relying solely on client-side validation are not acceptable production fixes for this endpoint.
index.ts (complete reproduction file)
// Local-only transport experiment. Never package or deploy this function.
Deno.serve(async (request) => {
const mode = new URL(request.url).searchParams.get('mode') ?? 'return';
if (request.method === 'OPTIONS') return new Response(null, {status: 204});
const response = () => new Response('too large', {status:413,headers:{'Content-Type':'text/plain','Connection':'close'}});
if (mode === 'return') return response();
if (mode === 'cancel') { await request.body?.cancel(); return response(); }
if (mode === 'cancel-nowait') { request.body?.cancel().catch(() => {}); return response(); }
if (mode === 'drain') { const reader = request.body?.getReader(); while (reader && !(await reader.read()).done) {} return response(); }
if (mode === 'bounded-drain') {
let readBytes = 0;
const reader = request.body?.getReader();
while (reader) {
const {done,value} = await reader.read();
if (done) break;
readBytes += value.length;
if (readBytes > 2*1024*1024+65536) { await reader.cancel(); break; }
}
return response();
}
if (mode === 'drain-response') {
return new Response(new ReadableStream({async start(controller) {
controller.enqueue(new TextEncoder().encode('too large'));
const reader = request.body?.getReader(); while (reader && !(await reader.read()).done) {}
controller.close();
}}), {status:413});
}
if (mode === 'response-stream') {
return new Response(new ReadableStream({start(controller) {
controller.enqueue(new TextEncoder().encode('too large'));
controller.close();
}}), {status:413});
}
if (mode === 'throw') throw new Error('too large');
return response();
});transport-probe.mjs (complete reproduction file)
import http from 'node:http';
const modes=process.argv.slice(2);
const byteCount=Number(process.env.PHOTO_PROBE_BYTES || (2*1024*1024+1));
if (!Number.isSafeInteger(byteCount) || byteCount<1 || byteCount>5*1024*1024) throw Error('Local probe size guard');
for (const mode of modes.length?modes:['return','cancel','cancel-nowait','drain','drain-response','response-stream']) {
const started=Date.now();
const result=await new Promise(resolve=>{
const body=Buffer.alloc(byteCount);
const req=http.request(`http://127.0.0.1:58121/functions/v1/photo-body-diagnostic?mode=${mode}`,{method:'POST',headers:{'Content-Type':'image/jpeg','Content-Length':body.length,Connection:'close'}},res=>{
console.log(JSON.stringify({mode,headersAt:Date.now()-started,status:res.statusCode}));
res.resume();res.on('end',()=>resolve({mode,status:res.statusCode,ms:Date.now()-started}));
});
req.on('error',e=>resolve({mode,error:e.message,ms:Date.now()-started}));
req.setTimeout(5000,()=>req.destroy(new Error('timeout')));
req.end(body);
});
console.log(JSON.stringify(result));
}
Activity
Luke-Rouse commentedon Sep 26, 2026
Update: Supabase CLI PR #6580
merged the same minimal correction on 14 September 2026. The local relay now
constructs the forwarded request from
reqrather thanreq.clone(), with anend-to-end regression covering an early HTTP 400 response while the request body
is left unread. The PR's reported CLI, stack and platform checks passed.
That resolves the proven local CLI forwarding cause, but it does not establish
that the managed Edge Functions relay uses or has deployed the same forwarding
change. Our hosted Beta checks previously reproduced both fixed-length and chunked
2 MiB + 1 boundary timeouts while the function should have returned HTTP 413.
Could a maintainer confirm whether the managed relay has received, or is scheduled
to receive, the equivalent non-cloning fix? If the hosted relay is owned outside
this repository, please route or transfer this report to the appropriate team. We
can rerun the existing boundary probes unchanged after deployment.
Luke-Rouse commentedon Oct 2, 2026
Fresh hosted regression on 2 October: authenticated fixed-length and chunked
2,097,153-byte uploads still time out after a 20-second client deadline instead
of delivering HTTP 413. Both finite 5 MiB variants also time out. We downloaded
the deployed function and verified its executable source against the reviewed
bounded handler before testing.
Controls pass: missing/invalid bearer return 401, exactly 2 MiB of invalid JPEG
input reaches validation and returns 415, normal JPEG upload preserves bytes,
authorised signing/delivery/removal work, and fixed/chunked 16 KiB+1 signing and
8 KiB+1 removal return 413. The focused matrix is 15/19 passed with four upload
responses unresolved. Every disposable fictional fixture was independently
removed; no credentials, account IDs, project references, real photos or private
content are included here.
The non-cloning correction in CLI PR 6580
is now included in stable 2.119.0. We also inspected the later
PR 6830 in
that exact tag:
it drains the incoming body abandoned by the worker before releasing the response.
That can make finite-body local probes pass, but the shown drain has no byte cap.
We have not run a new normal-runtime acceptance test against that release, and
do not claim it proves the managed implementation or bounded hosted rejection.
Can a maintainer confirm the managed relay's ownership/deployment status and
provide a supported way to deliver early 413 while preserving byte/time bounds
on rejected input? Please route this report to the managed Edge Functions team
if needed. Increasing the accepted limit, unrestricted application-body draining,
patching generated runtime code or relying only on client validation would not
meet this endpoint's acceptance criteria. We can rerun the same bounded hosted
probes after a correction.
emerginginv commentedon Oct 3, 2026
Fresh hosted reproduction on 2 October 2026 from a Stripe webhook endpoint with a strict 1 MiB raw-body limit.
All requests were finite, at most 2 MiB, and used deliberately invalid or missing Stripe signatures. No authenticated billing operations or customer fixtures were involved.
Hosted results with a 10-second client deadline:
We also tested the unchanged production body reader in an isolated Edge Runtime v1.74.1 container, with no database or credentials. The main worker used
EdgeRuntime.applySupabaseTagand forwarded to a user worker:Ten local body-boundary tests pass, including preservation of signed Unicode/whitespace payloads below and exactly at the limit. Removing application cancellation was previously tested and did not resolve the cloned-forwarding failure.
This isolates forwarding ownership in the local reproduction. We cannot inspect the hosted relay, so the identical hosted cause remains an inference.
Could the managed Edge Functions team confirm whether the hosted forwarding path has received the non-cloning correction from CLI PR #6580, or provide a supported way to release early responses with strict byte/time bounds? Unrestricted draining or increasing the accepted body limit would undermine the security boundary. We can rerun the same hosted matrix after a provider correction.
This report contains no credentials, project references, customer data, or private request identifiers.
cemavega commentedon Oct 4, 2026
We are seeing a closely related boundary failure on hosted Supabase Edge Functions, with SupabaseEdgeRuntime 1.76.0 identified in correlated logs.
Our hosted observations are:
We have not established that the hosted relay uses the same implementation described in #737, but the observed boundary is strikingly similar.
Could Supabase confirm:
We stopped further timeout probing and have preserved the existing security and input limits.