Re-review at e625e9f3. The delta from 3f1fcf37 has no behaviour change:

• three comments removed for the ratchet;
• shownSizeScaleFilter renamed to halveToCoverScaleFilter;
• -- added before the frame path in two test ffprobe calls.

One leftover nit: the old /** Extracts the given frames of deferred videos… */ JSDoc now documents shownDeferredFrameIndexes, so the two docs are effectively swapped. My review below stands.

Reviewed at 3f1fcf37. No blockers; approving once required CI is green.

• The measured box is conservative in every direction I could find.
  • getBoundingClientRect() × devicePixelRatio includes transforms, and a rotated element's bounding rect is larger than its drawn size.
  • The filter uses max(bw/iw, bh/ih), which is the right scale for cover and an over-estimate for contain.
  • Halving stops while the frame still covers the box, rounds sides up to even, and never upscales.
  • A video with a zero-size rect at every sampled time (display:none, never laid out) is filtered out and stays full size.
• Fail-safe paths keep full size:
  • grading attribute, object-fit: none | scale-down, or any non-grading <canvas> at a sampled time;
  • missing __hf.seek;
  • the parallel-BeginFrame branch, where shownBoxes stays undefined;
  • HDR, PNG and alpha frames, because the filter is only pushed in the SDR jpg branch.
• It's exact per chunk: each chunk measures at exactly its own frame times, so a zoom crossing chunks is full size where it's big. I checked that chunk range extraction (extractVideoFramesRange) doesn't go through the content-keyed extract cache (that's only extractAllVideoFrames), so a shrunk extraction can't be reused by another chunk or render that needs full size.
• Same inputs to session boot. skipReadinessVideoIds and the remote metadata hints are now derived from the plan (shownDeferredFrameIndexes, and deferred entries ∩ remote IDs). They give the same sets as the old extraction-output-derived ones, since a deferred video with no shown frames had an empty framePaths. Remote videos still get their plan-sized <video>, so the measured rect isn't a 300×150 default.

Non-blocking:

1. An optimization that can fail a chunk. measureVideoShownBoxes throwing, from a page.evaluate protocol timeout or an exception in a seek, now fails the chunk. Before, this step didn't exist. Wrapping it with catch → undefined (full size, plus a perf-sidecar note) would make the optimization fail open.
2. Canvas detection only sees DOM canvases. An OffscreenCanvas, or a canvas created but never appended, that samples a video (drawImage → toDataURL into an <img>) isn't detected, so it would read a halved frame. That's rare, but worth a line in the JSDoc.
3. Misplaced doc comment. In chunkFrames.ts, the old /** Extracts the given frames of deferred videos… */ now sits directly above shownDeferredFrameIndexes's own JSDoc, so extractDeferredVideoFramesForChunk lost its doc. Moving it back also helps the non-required Comments ratchet, which is red on chunkFrames.ts (+2 lines) and videoFrameExtractor.ts (+3).
4. Extra seek pass. The measurement adds one seek per chunk frame before capture. Your fixture shows a net win; on a page with a costly seek and only a full-frame video, it's pure overhead. Skipping the pass when every shown deferred video's plan-time size already fits the viewport would avoid it.

— Somu

Reviewed at e625e9f3. There are no reviews on GitHub yet. Somu's thread pre-review at 3f1fcf37 covered the escape hatches and raised three non-blocking points (a measure throw fails the chunk, OffscreenCanvas, and the moved JSDoc); I'm not repeating them.

What holds

• The measured box is conservative. It is getBoundingClientRect × devicePixelRatio, taken over every frame of the chunk, and it agrees with capture: the viewport DSF and clip.scale are both deviceScaleFactor (screenshotService.ts:227-228, frameCapture.ts:1526-1530). The halving fit max(W/iw, H/ih) is the cover scale, so it is never too small for contain or fill either. The parallel BeginFrame branch never measures, so it stays full size.
• The canvas check returns on the first seek, so pages with incremental canvas or rAF drawing are never seeked out of order before capture.
• Verified locally with ffmpeg 7.0.2 and headless Chrome:
  • the new extractor test and both shownBoxes browser tests pass;
  • renderChunk.test.ts gives 28/29 here, and the shown-size chunk test runs rather than skips. The one failure is the byte-range test's served.bytes < size/3; it flakes the same way on the base e44f456b, and CI's Test is green.
• Mutation checks (each turns tests red): the extractor ignoring the box, measuring without DPR, dropping the canvas check, dropping the grading check, and dropping the object-fit: none | scale-down check.

Blocker: rounding to even changes the aspect, which draws a 1 px background column on each side of a default-fit video (videoFrameExtractor.ts:271, 2*ceil(${dim}/pow(2, halvings)/2))

• 1080p and 4K sources halve to an odd height once the box is under 240 device px wide: 1920x1080 → 240x135, which becomes 240x136. With real ffmpeg the filter gives 240x136 for a 1920x1080 or 3840x2160 source and a 200x112 or 230x130 box.

• The replacement <img> copies the video's object-fit (screenshotService.ts:802), and a <video>'s default is contain (I checked getComputedStyle in the headless shell). A 240x136 frame in a 16:9 box is letterboxed.

• Repro in Chrome: a white frame drawn into a 200x112.5 object-fit: contain box, minimum value of the edge columns (0, 1, 198, 199):

  frame	edge columns
  1920x1080 (main)	255, 255, 255, 255
  240x136 (this PR)	0, 255, 255, 0
  240x135	255, 255, 255, 255

  So a small camera bubble from a 1080p or 4K source, with the default fit, gets a full-strength column of whatever is behind it on both sides, on every frame. That is the PR's target case, and "picture unchanged" doesn't hold for it. A 1 px column is too small to fall below the 0.97 in-box SSIM bar, which would explain why the fixtures didn't catch it. cover and fill are unaffected.

• Fix: drop the even rounding, ceil(${dim}/pow(2, halvings)). The JPEG path doesn't need even sizes: the same filter chain with /8 encodes 240x135 yuvj420p cleanly, and 1080p, 4K and 720p sources then halve exactly. With that change the extractor test gets 16x9 for the 64x36 source instead of 16x10, so its expectation would change with it.

Non-blocking

• No test checks that renderChunk passes the measured boxes on to extraction. With shownBoxes: undefined at renderChunk.ts:788 the chunk suite still passes, because the shown-size test only checks the sidecar, which records the measurement whether or not extraction used it, and that fixture renders png-sequence. Asserting a deferred frame's decoded size in that test would pin the wiring.

CI: the eight required checks are green at e625e9f3.

Verdict: REQUEST CHANGES
Reasoning: The measurement and the escape hatches are sound and pinned by tests, but rounding the halved size to even letterboxes default-fit videos from 1080p and 4K sources by a pixel on each side. The fix is a one-line change plus the test expectation.

— Via