iframe-proxy

JulianCloudNTH · 2026-06-13T00:08:56Z

Stack from ghstack (oldest at bottom):

[ExecuTorch][WebGPU] et_vk.prepack test suite (export + native golden) #20292
-> [ExecuTorch][WebGPU] Add et_vk.prepack (constant-tensor packing) for E2E weight loading #20265
[ExecuTorch][WebGPU] et_vk.apply_rotary_emb test suite (export + native golden) #20290
[ExecuTorch][WebGPU] Add et_vk.apply_rotary_emb (interleaved RoPE) + ValueList multi-output #20264
[ExecuTorch][WebGPU] et_vk.embedding_q4gsw test suite (export + native golden) #20289
[ExecuTorch][WebGPU] Add et_vk.embedding_q4gsw (4-bit groupwise-symmetric quantized embedding) #20263

Adds the WebGPU backend handler for et_vk.prepack.default, the node the VulkanPartitioner wraps around every constant feeding a delegated op so the constant is materialized into its dedicated GPU buffer before inference.

For the WebGPU backend's buffer-flat/fp32 model, prepack is an identity layout (same dims, dtype, and bytes), so the handler runs no compute shader: it validates that src and out match (dims, elem_size, nbytes, non-null buffers; every check throws fail-loud) and records a one-time src->out buffer-to-buffer copy via the new WebGPUGraph::add_prepack_copy. The recorded copies run once in a new build() Phase 4 (after the op-dispatch chain is recorded), mirroring the Vulkan delegate's separate prepack() init phase (distinct from per-inference execute()). Ordering is guaranteed by the WebGPU queue -- the prepack submit precedes the first execute() submit on the same queue, so the copied data is visible without an explicit device poll (Dawn has no wgpuDevicePoll, and the backend relies on queue ordering plus the output-map wait elsewhere).

src.elem_size is the WebGPUTensor field added by the embedding op lower in this stack, so prepack stacks above it.
@exported-using-ghexport

Differential Revision: D108428754

[ghstack-poisoned]

pytorch-bot · 2026-06-13T00:08:59Z

github-actions · 2026-06-13T00:09:34Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[ghstack-poisoned]

…E2E weight loading Pull Request resolved: #20265 Adds the WebGPU backend handler for `et_vk.prepack.default`, the node the VulkanPartitioner wraps around every constant feeding a delegated op so the constant is materialized into its dedicated GPU buffer before inference. For the WebGPU backend's buffer-flat/fp32 model, prepack is an identity layout (same dims, dtype, and bytes), so the handler runs no compute shader: it validates that `src` and `out` match (dims, `elem_size`, `nbytes`, non-null buffers; every check throws fail-loud) and records a one-time `src`->`out` buffer-to-buffer copy via the new `WebGPUGraph::add_prepack_copy`. The recorded copies run once in a new `build()` Phase 4 (after the op-dispatch chain is recorded), mirroring the Vulkan delegate's separate `prepack()` init phase (distinct from per-inference `execute()`). Ordering is guaranteed by the WebGPU queue -- the prepack submit precedes the first `execute()` submit on the same queue, so the copied data is visible without an explicit device poll (Dawn has no `wgpuDevicePoll`, and the backend relies on queue ordering plus the output-map wait elsewhere). `src.elem_size` is the `WebGPUTensor` field added by the embedding op lower in this stack, so prepack stacks above it. ghstack-source-id: 393506814 @exported-using-ghexport Differential Revision: [D108428754](https://our.internmc.facebook.com/intern/diff/D108428754/)

[ghstack-poisoned]

JulianCloudNTH · 2026-06-16T04:53:33Z

@claude review

claude · 2026-06-16T04:54:24Z

Claude finished @JulianCloudNTH's task in 2m 23s —— View job

Code Review: WebGPU `et_vk.prepack` handler

Read PR diff against base
Analyze WebGPUGraph build/prescan/materialize logic
Analyze Prepack.cpp handler
Check correctness, buffer usage, ordering, and memory accounting

Overall this is a clean implementation and the deferred-constant design is nicer than what the PR text describes. A few things worth addressing before merge.

1. PR description doesn't match the implementation

The description says prepack records a src→out buffer-to-buffer copy via WebGPUGraph::add_prepack_copy, run in a new build() Phase 4. None of that exists in the diff:

There is no add_prepack_copy method.
There is no Phase 4 — the copies happen inline during the Phase 3 op-dispatch loop (WebGPUGraph.cpp:531).
It's not a buffer→buffer copy. Deferred constants get no eager src buffer; prepack_impl materializes the .pte bytes directly into the out buffer via wgpuQueueWriteBuffer (materialize_constant, WebGPUGraph.cpp:543).

The actual approach (defer + single CPU→GPU write into the consumer buffer) is better — it avoids the redundant src buffer and the extra copy. Please update the PR/commit description so reviewers and git log reflect what landed.

2. Duplicated constant-loading logic

The inline-offset / named-data-map load is now written twice — the eager path (WebGPUGraph.cpp:330-395) and materialize_constant (WebGPUGraph.cpp:543-575) implement the same offset() != UINT64_MAX vs named_key + undersized-check + Free() logic. This is ~20 lines of non-trivial duplication that can drift. Consider extracting a single helper write_constant_to(WGPUBuffer dst, const ConstantSource&) (or have the eager path build a ConstantSource and call materialize_constant), which would also unify the error messages. Fix this →

3. Confirm prepack `out` buffers are never memory-plan aliased

materialize_constant writes into out.buffer once at build time. If out ever has a mem_obj_id >= 0 (shared buffer), a later execute() write to a transient aliasing the same mem_obj_id would clobber the prepacked weights. This is presumably safe because the partitioner gives prepacked weights dedicated, non-planned buffers — but the invariant isn't asserted anywhere. Worth a one-line comment (or assert) in prepack_impl documenting that out must be a dedicated buffer, since the correctness of write-once depends on it.

Smaller notes

out.buffer usage flags: the direct wgpuQueueWriteBuffer into out requires CopyDst. Both the dedicated (WebGPUGraph.cpp:319) and shared (:469) buffer paths set CopyDst, so this is fine — just flagging it as a real dependency of the deferred design.
Prescan allocations: oc->name()->str() == "et_vk.prepack.default" (:258) builds a std::string per op per chain pass. Minor, but oc->name()->string_view()/c_str() comparison avoids the allocation in the hot prescan loop.
Deferred-constant invariant: correctness hinges on a deferred constant (null buffer) being reachable only through prepack. The prescan enforces this via direct_use_ids + ValueList expansion (:267-277), which looks correct. Just note it doesn't account for a constant appearing directly in graph->output_ids() — almost certainly impossible for a prepack source, but it's the one path not covered by the prescan.
memory_stats fix (:891) correctly excludes deferred (null-buffer) constants. 👍

Correctness

The defer logic, source recording, and ordering (queue-ordered writeBuffer before the first execute submit) are sound. No blocking bugs found — items 1–3 above are the main things to resolve, and #1 (description) is purely cosmetic.
· branch gh/JulianCloudNTH/27/head

[ghstack-poisoned]

#20292) Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * __->__ #20292 * #20265 * #20290 * #20264 * #20289 * #20263 Test suite for the `et_vk.prepack` constant-materialization op, split into its own diff (op below, tests above) per the per-op test-split convention. The prepack op is how a serialized constant becomes a GPU tensor: the constant arrives as a CPU-side reference (sizes + a pointer into the .pte bytes), and the prepack node is the sole materialization — one CPU->GPU transfer straight into the consumer's buffer. The model `M(x) = x + w` (w a constant) routes `w` through a prepack node, so the delegate must run the materialization for the output to equal `x + w` rather than `x + 0`. @exported-using-ghexport Differential Revision: [D108678631](https://our.internmc.facebook.com/intern/diff/D108678631/) Differential Revision: [D108678631](https://our.internmc.facebook.com/intern/diff/D108678631)

…E2E weight loading Pull Request resolved: #20265 Adds the WebGPU backend handler for `et_vk.prepack.default`, the node the VulkanPartitioner wraps around every constant feeding a delegated op so the constant is materialized into its dedicated GPU buffer before inference. For the WebGPU backend's buffer-flat/fp32 model, prepack is an identity layout (same dims, dtype, and bytes), so the handler runs no compute shader: it validates that `src` and `out` match (dims, `elem_size`, `nbytes`, non-null buffers; every check throws fail-loud) and records a one-time `src`->`out` buffer-to-buffer copy via the new `WebGPUGraph::add_prepack_copy`. The recorded copies run once in a new `build()` Phase 4 (after the op-dispatch chain is recorded), mirroring the Vulkan delegate's separate `prepack()` init phase (distinct from per-inference `execute()`). Ordering is guaranteed by the WebGPU queue -- the prepack submit precedes the first `execute()` submit on the same queue, so the copied data is visible without an explicit device poll (Dawn has no `wgpuDevicePoll`, and the backend relies on queue ordering plus the output-map wait elsewhere). `src.elem_size` is the `WebGPUTensor` field added by the embedding op lower in this stack, so prepack stacks above it. ghstack-source-id: 395549289 @exported-using-ghexport Differential Revision: [D108428754](https://our.internmc.facebook.com/intern/diff/D108428754/)

Update

bf5c220

[ghstack-poisoned]

JulianCloudNTH requested review from kirklandsign and larryliu0820 as code owners June 13, 2026 00:08

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 13, 2026

Update

76c146d

[ghstack-poisoned]

JulianCloudNTH temporarily deployed to cadence June 15, 2026 17:23 — with GitHub Actions Inactive

meta-codesync Bot added the meta-exported label Jun 15, 2026

Update

b1de57c

[ghstack-poisoned]

JulianCloudNTH mentioned this pull request Jun 15, 2026

[ExecuTorch][WebGPU] et_vk.embedding_q4gsw test suite (export + native golden) #20289

Merged

JulianCloudNTH temporarily deployed to cadence June 15, 2026 23:00 — with GitHub Actions Inactive

This was referenced Jun 15, 2026

[ExecuTorch][WebGPU] et_vk.apply_rotary_emb test suite (export + native golden) #20290

Merged

[ExecuTorch][WebGPU] et_vk.prepack test suite (export + native golden) #20292

Merged

Update

260a038

[ghstack-poisoned]

JulianCloudNTH temporarily deployed to cadence June 16, 2026 05:18 — with GitHub Actions Inactive

psiddh approved these changes Jun 21, 2026

View reviewed changes

Update

69ed368

[ghstack-poisoned]

JulianCloudNTH temporarily deployed to cadence June 22, 2026 05:06 — with GitHub Actions Inactive

meta-codesync Bot merged commit 7a66f0a into gh/JulianCloudNTH/27/base Jun 22, 2026
176 of 183 checks passed

meta-codesync Bot deleted the gh/JulianCloudNTH/27/head branch June 22, 2026 16:12

meta-codesync Bot temporarily deployed to cherry-pick-bot June 22, 2026 16:12 Inactive

pytorchbot mentioned this pull request Jun 22, 2026

[ExecuTorch][WebGPU] Add et_vk.prepack (constant-tensor packing) for E2E weight loading #20429

Merged

Sunbelt Computer Software

PL/B Language Development and Support

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ExecuTorch][WebGPU] Add et_vk.prepack (constant-tensor packing) for E2E weight loading#20265

[ExecuTorch][WebGPU] Add et_vk.prepack (constant-tensor packing) for E2E weight loading#20265
meta-codesync[bot] merged 5 commits into
gh/JulianCloudNTH/27/basefrom
gh/JulianCloudNTH/27/head

JulianCloudNTH commented Jun 13, 2026 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Jun 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 13, 2026

Uh oh!

JulianCloudNTH commented Jun 16, 2026

Uh oh!

claude Bot commented Jun 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Conversation

JulianCloudNTH commented Jun 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jun 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20265

❗ 1 Active SEVs

❌ 3 New Failures, 3 Unrelated Failures

Uh oh!

github-actions Bot commented Jun 13, 2026

This PR needs a release notes: label

Uh oh!

JulianCloudNTH commented Jun 16, 2026

Uh oh!

claude Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review: WebGPU et_vk.prepack handler

1. PR description doesn't match the implementation

2. Duplicated constant-loading logic

3. Confirm prepack out buffers are never memory-plan aliased

Smaller notes

Correctness

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JulianCloudNTH commented Jun 13, 2026 •

edited

Loading

pytorch-bot Bot commented Jun 13, 2026 •

edited

Loading

This PR needs a `release notes:` label

claude Bot commented Jun 16, 2026 •

edited

Loading

Code Review: WebGPU `et_vk.prepack` handler

3. Confirm prepack `out` buffers are never memory-plan aliased