feat(core,engine): track accelerated canvases before any page script runs by vanceingalls · Pull Request #5970 · heygen-com/hyperframes · GitHub
Skip to content

feat(core,engine): track accelerated canvases before any page script runs - #5970

Open
vanceingalls wants to merge 1 commit into
feat/capture-blur-defectsfrom
feat/capture-blur-scan
Open

vanceingalls wants to merge 1 commit into
feat/capture-blur-defectsfrom
feat/capture-blur-scan

Conversation

@vanceingalls

@vanceingalls vanceingalls commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

What

Makes the tracker that spots WebGL / WebGPU canvases see the cases that slipped past it, and counts the frames that fell back to plain DOM.

  • The accelerated-canvas tracker (pageMotionBlur.ts) is now a self-contained function. Besides wrapping getContext, it wraps transferControlToOffscreen: a canvas handed to an OffscreenCanvas creates its WebGL context out of reach, so a transferred canvas counts as accelerated. Its registry lives on window, so independent installs agree.
  • It runs before any author script. The engine evaluates it on every new document (installPageMotionBlur); insertRuntimeTag, used by the bundler and Studio's preview, puts it at the very start of <head> for pages that opt into blur; the runtime's own install is idempotent with both. Before, it was installed when the runtime evaluated, so a context made by an earlier author script was missed. Either miss left a frozen (Chrome 152) or black (Chrome 157) capture with blur still on.
  • report() carries failedFrames, the cumulative count of frames that rendered plain DOM because their capture failed or timed out. failures keeps its streak meaning (it resets after a success), so an "on" report can no longer hide a render where captures keep failing.
  • Core's unpacked size budget goes from 3,903,211 to 3,930,000 bytes. With the lifecycle, the compositor and the tracker, core measures 3,903,549 here, just over the old budget; the rest of the stack needs about 3,925,300 at its tip, so the ceiling is raised once, with about 4.7 KB of headroom, rather than in several PRs.

Why

Part 5 of 9 in the capture motion blur stack: #5875 → #5967 → #5968 → #5969 → #5970 → #5971 → #5990 → #5991 → #5992. It builds on #5969, which added installPageMotionBlur in the engine.

Stack goal: opt-in capture motion blur for compositions, in Studio preview and in renders, drawn with Chrome's html-in-canvas. A frozen or black canvas inside a "blurred" frame is a wrong frame, so the runtime has to know about every accelerated canvas, whenever it was created.

Notes

Testing

  • Core pageMotionBlur.test.ts: the tracker records getContext WebGL / WebGPU canvases and transferred canvases, is idempotent across installs, and its source runs standalone.
  • Core htmlDocument.test.ts: the tracker goes at the start of <head> for a page that opts into blur, once, and not for a page that does not.
  • Core captureMotionBlur/index.test.ts: blur stays off for a canvas handed to an OffscreenCanvas; failedFrames counts every fallback frame though the failure streak resets; engine pageMotionBlur.test.ts: the tracker is the first script installed.
  • bun run build, tsc --noEmit in core and engine, and their tests pass; comment ratchet against feat(producer,studio): wire capture motion blur into renders and Studio #5969's branch passes (the tracker's doc comment is two lines, to stay at the file's comment share); every package is within its packed and unpacked size budget.

🤖 Generated with Claude Code

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta review at full head8bb0fe5ebc20dbf978c01d3ad3dc9384c73f2c85, relative tofcf0512a.

Audited the t=0 discovery baseline, per-step defect accumulation before wrapping, normal-path time restoration and timeline invalidation; document-wide GPU check; HDR/layered route reason through buildCaptureOptions and engine installation. Route-off reasons preserve explicit preview-off when absent. The240-step scan/backface-every10 sampling boundary remains bounded, and this change does not promise pixel identity across workers.

The inherited #5875 pose and #5967 timeout findings remain; this delta approval does not clear the complete stack. Saved/fresh prior human reviews and inline comments were empty. Exact-head hosted checks pass/skip; the private focused core run passed124 authored tests. Inspected child regression logs execute3 visual/audio cases, not every new unit test. Browser-version screenshots, HDR output, worker parity, colour and performance measurements were not independently rerun. No provider call, merge or release.

— Magi (Home)
Verdict: APPROVE
Reasoning: No additional blocking defect found in this change relative to its stack parent; the foundation fixes remain required.

@vanceingalls
vanceingalls marked this pull request as draft October 20, 2026 21:58
@vanceingalls
vanceingalls force-pushed the feat/capture-blur-defects branch from fcf0512 to af9d2eb Compare October 21, 2026 01:55
@vanceingalls
vanceingalls force-pushed the feat/capture-blur-scan branch from 8bb0fe5 to 4e120e5 Compare October 21, 2026 01:55
@vanceingalls vanceingalls changed the title fix: decide blur defects once per timeline, off on the HDR route feat(core,engine): track accelerated canvases before any page script runs Oct 21, 2026
@vanceingalls
vanceingalls force-pushed the feat/capture-blur-defects branch from af9d2eb to 9ec57dd Compare October 21, 2026 03:53
@vanceingalls
vanceingalls force-pushed the feat/capture-blur-scan branch from 4e120e5 to 793f8a3 Compare October 21, 2026 03:53
@vanceingalls
vanceingalls marked this pull request as ready for review October 21, 2026 04:16
…runs

html-in-canvas captures a WebGL or WebGPU canvas frozen or black, so the
runtime turns blur off when the composition draws one. Two cases slipped
past the tracker that spots them: a canvas handed to
transferControlToOffscreen (its context is created out of reach) and a
context made by an author script that runs before the runtime.

- The tracker is a self-contained function that also wraps
  transferControlToOffscreen (a transferred canvas counts as accelerated)
  and keeps its registry on window so independent installs agree.
- The engine evaluates it on every new document, ahead of any page script.
- insertRuntimeTag (bundler and Studio preview) puts it at the very start of
  <head> for pages that opt into blur; the runtime install is idempotent
  with it.
- report() carries failedFrames, the cumulative count of frames that
  rendered plain DOM because their capture failed or timed out.
  failures keeps its streak meaning.

Core's unpacked size budget rises from 3903211 to 3930000 bytes: the runtime
lifecycle, the compositor and the tracker together pass the old ceiling.

Folded review fixes: R6, R17 (the report counter; the engine log line lands
with the runtime's reason codes).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@vanceingalls
vanceingalls force-pushed the feat/capture-blur-defects branch from 9ec57dd to 774f8ab Compare October 21, 2026 05:18
@vanceingalls
vanceingalls force-pushed the feat/capture-blur-scan branch from 793f8a3 to d8d7200 Compare October 21, 2026 05:18

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head d8d72006e0d837d70a7d501dd65473bd4af2a8a3 relative to 774f8abccd4d0290d2c28257d259ad435a15145d.

Audited early compiler/render tracking before author scripts, idempotent getContext wrapping, offscreen transfer and package exports. The tracker is shared through the window WeakSet. The core unpacked budget really increases3903211→3930000; it is not byte-neutral. Local tracker controls pass; author package-size/build evidence remains distinct from a fresh reviewer build.

Independent revision-bound review of the nine-PR composition. Earlier approvals/requests covered older heads. Current saved head/review/inline/check snapshot has no fresh duplicate finding and no failing/pending latest check. Stacked broad-suite skips are not executed tests.

Audited: Early idempotent accelerated-canvas tracking in compiler/engine before authored scripts, OffscreenCanvas transfer, package exports and the explicit core unpacked budget change.
Limits: Composed local focused run:196 authored core tests pass; separate WAAPI/CLI44 pass. Synthetic geometry/canvas controls, not fresh browser pixel/performance proof. No app/model/provider call, source fix, merge or release.

— Magi / desktop-review

Verdict: APPROVE
Reasoning: No additional blocker in this own delta. The later #5971 defect must be repaired before landing the complete stack.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants