Repository navigation
perf(producer): multi-worker mp4 renders start encoding while frames are still being captured - #5952
Conversation
…are still being captured On the multi-worker disk capture path the encoder used to start only after every worker had finished. It now starts with capture and is fed frame 0, 1, 2... in order as each frame file lands, with the same encoder arguments, so the output is byte-identical and the encode left after capture is shorter. Off for drawElement capture (a failed self-check discards every frame) and for png/gif/webm/mov/HDR/GPU/chunked/HLS output. If the overlapped encode fails, the frames are encoded after capture exactly as before.
…he encoder With every core busy capturing, the encoder only slowed capture down, so the overlap now runs only when the render uses fewer workers than the CPU count worker sizing sees. Also pins the reviewed producer test-lane source hash and drops an added comment line.
jrusso1020
left a comment
There was a problem hiding this comment.
Reviewed exact a6db722f33e85e2dc17eb119aee215d7add25071. No blocking finding.
packages/producer/src/services/render/stages/encodeStage.ts:320–426 consumes landed frames in index order, falls back from an atomically renamed worker path to the merged path, and serially re-encodes after overlap failure. The orchestrator aborts and awaits the overlap encoder on capture failure. Shared videoEncoderOptions preserves encoder settings, and the eligibility gate excludes drawElement recapture, GPU/HDR, chunk/HLS and incompatible formats. Adaptive retries retain the frames already reported rather than invalidating fed frames.
The fresh adversarial pass ran the original four real-ffmpeg integration tests plus three additional controls: corrupted landed input recovers through the serial path with matching SHA, fractional 30000/1001 FPS retains byte identity, and documented exclusions stay excluded. Seven controls pass. Local testing used exact-source export aliases/narrow core and engine facades because generated runtime dependencies were unavailable; encoder, process and filesystem implementations were not mocked. These are focused encode tests, not a complete browser render or cross-platform acceptance. The author’s broader render timing measurements remain author-reported.
Audited: new overlap owner, eligibility/options extraction, changed coordinator/frame reporting, capture/adaptive-retry/merge and orchestrator finish/abort integration, test registration and tests. Trusting: unchanged large orchestrator branches outside this path and browser capture internals. No existing substantive review findings to duplicate.
Need: yes · Size: coherent capture/encode overlap · Reuse: existing streaming encoder and shared disk encoder options · Simpler: sequential indexed feeding with serial fallback; no new capture scheduler.
— hf-review
Verdict: APPROVE
Reasoning: Frame ordering, merge races, capture failure cleanup and serial fallback hold in source review and focused real-encoder controls.

What
On a multi-worker mp4 render that captures frames to disk, the encoder now starts with capture instead of after it. It is fed frame 0, 1, 2... strictly in order as each frame file lands, with the same encoder arguments as the after-capture encode, so the output is byte-identical and less encode is left once capture ends.
Why
On this path capture and encode ran one after the other: the encoder only started once every worker had finished, so a render's wall clock was roughly capture plus encode. The streaming path already overlaps them, but the multi-worker streaming mode stays off by default for screenshot capture, so a large share of multi-worker renders took the serial path.
Related work
Refs the default-off multi-worker streaming path (
HF_CAPTURE_PARALLEL_STREAM), which this does not change.How
ParallelProgress.landedFrame). The write is synchronous and completes before the report, so a reported frame is never partial.encodeWhileCapturing(encode stage) spawns the existing streaming encoder at capture start and feeds frames in index order: it waits for frame N to be reported, reads it from the worker's dir, or from the merged frames dir if the post-batch merge has already renamed it (rename is atomic, so the frame is in one place or the other). Once capture succeeds it reads any remaining frames from the merged dir and closes the encoder.videoEncoderOptions, now shared). For software libx264 from JPEG frames, image2pipe and the image-sequence input decode the same JPEG bytes into the same frames, so the bitstream is identical.cpuCountcomputeWorkerSizingreports). With every core capturing, the encoder only slows capture down, so the render keeps today's serial encode.Test plan
encodeWhileCapturing.test.ts(integration lane, real ffmpeg, preset medium crf 16). Frames land out of order across two workers' dirs and must be encoded in index order, byte-identical torunEncodeStageon the same frames; frames already moved by the merge, or never reported, are still read;abortstops an encoder that is waiting on frames. Each test failed with the change broken: with the sources at main,encodeWhileCapturing is not a function; when feeding only after capture, the overlap test times out (condition not reached); with one encoder argument changed (crf 17),expected '29745d05…' to be '135a554a…'. A gate test (4 workers with 8 cores runs the overlap; 4 workers with 4 cores and 1 worker do not) fails withexpected true to be falseonce the spare-core check is removed. It passed 3 runs in a row.With the spare-core gate:
The CPU count worker sizing reads is the machine's core count. It does not see a container CPU quota: the Linux box above caps each job at 4 CPUs through a cgroup quota, and sizing still sees 8. So on a quota-limited container the gate uses the same core count sizing does.
Byte identity: the 8 s and 142 s films produce the same output sha on main and on this branch in every run. The two other video fixtures are not byte-stable run to run even on main (capture varies), so for all four I also ran today's encode on the very frames each overlapped render used. The video-only output matched in every case (8 s
1af06eca, 17 sf7142e7b, 42 s60d81447, 142 s7afb62fb), and the final mp4 shas for the 17 s film on this branch fall in the same set main produces.Not run: macOS and Windows renders (no reachable machine), HLS, and the drawElement disk path (excluded by design).