Preview: a trimmed audio clip is heard past its end, in a hidden tab (transport stalls) and at every buffer end (lagging element unmuted) · Issue #5710 · heygen-com/hyperframes · GitHub
Skip to content

Preview: a trimmed audio clip is heard past its end, in a hidden tab (transport stalls) and at every buffer end (lagging element unmuted) #5710

Closed

Description

@sontiac

Previewing in <hyperframes-player> a composition that plays one source file as a series of trimmed clips (<audio data-start data-duration data-media-start> per kept segment of a talking-head take), parts of the take are heard that the edit removed. We found two causes. Both are in 0.8.82, and the code of both is unchanged on main (checked at v0.8.165).

1. In a hidden tab, the transport stops while media plays on. The transport ticks on requestAnimationFrame only (scheduleNextTransportFrame, wakeTransport and the initial tick in runtime/init.ts). A hidden document runs no animation frames, but its media keeps playing, so syncRuntimeMedia stops running while it does. Nothing pauses an element at the end of its clip window, and the next clip never starts, until the tab is shown again. A person listening to a draft in another tab hears the raw take, retakes included.

Reproduction: two clips from one source file, e.g. a1 (data-start="0" data-duration="0.733" data-media-start="0.166") and a2 (data-start="0.733" data-duration="1.467" data-media-start="1.166"). Play, then switch tabs during a1. a1 plays to the end of its source, and a2 never starts. In a test, holding the document's animation frames reproduces it deterministically.

Proposed fix: while document.visibilityState === "hidden", schedule the transport tick with a timer (16 ms) instead of requestAnimationFrame. On visibilitychange, move a pending tick to whichever scheduler the new state calls for; a frame requested just before the tab hid would otherwise never fire. Chromium doesn't throttle the timers of a page that is playing audio, so the hidden tick keeps time. A paused, hidden transport still parks as it does today. The change is about 30 lines in init.ts: requestTransportTick, cancelTransportTick and a visibilitychange listener registered with the runtime's cleanup callbacks, used in place of the four requestAnimationFrame(transportTick) / cancelAnimationFrame calls on state.transportRafId.

2. When a clip's Web Audio buffer ends, its element is unmuted while it still plays. WebAudioTransport plays a decoded clip as an AudioBufferSourceNode bounded to its window, and mutes the clip's element, which syncRuntimeMedia keeps playing beside the buffer. The element lags the buffer: we measured 20–75 ms in Brave (Chromium 154). On the buffer's natural ended, handleSourceEnded restores el.muted = priorMuted while the element is still playing, so the lag is replayed aloud until a tick pauses the element: 1–17 ms at a cut between clips, and 70–100 ms at the last clip, where no other buffer holds the clock. It happens with the tab in front too; in a hidden tab it is often longer, since Chromium repeatedly pauses the muted element and the runtime restarts it. In the 24 trials we recorded, the replay happened at every last-clip end.

Proposed fix: on a natural end, keep a non-looping element muted, and owned for ownsElement (so syncRuntimeMedia's per-tick mute keeps applying), until the next stopAll(), which every pause, seek and end of playback already calls; a new claim of the element first restores it. A looping element is restored at once as today, since its buffer doesn't loop. About 30 lines in webAudioTransport.ts (a _finished map, handleSourceEnded, stopAll, ownsElement and a releaseFinished called by both schedule paths), with unit tests in webAudioTransport.test.ts; the existing runtime tests pass unchanged except the natural-end test, which asserted the unmute.

We run both as a patch on 0.8.82 in our studio, each with a browser test of its bug (the first fails without its fix every time; the second, a timing race, failed 5 of 6 runs without it and passes with it). Happy to open a PR (or two, one per cause) if the approach suits you.

Activity

  1. krishs0404 commented on Oct 19, 2026

    @krishs0404
    Contributor

    Hi, I'd like to take cause 1 (the transport stalling in a hidden tab) as a PR, for my Stanford CS146S open source contribution. I'll follow the approach in the report: a requestTransportTick / cancelTransportTick pair in runtime/init.ts that uses a 16 ms timer while document.visibilityState === "hidden" and requestAnimationFrame otherwise, a visibilitychange listener that moves a pending tick to the right scheduler (removed in teardown), and a runtime test that holds animation frames to reproduce the stall. A paused, hidden transport still parks as today.

    I'm leaving cause 2 (webAudioTransport.ts, the unmute on natural end) alone, so the reporter can open that as a separate PR if they still want to. Could a maintainer confirm the claim?

  2. jrusso1020 commented on Oct 23, 2026

    @jrusso1020
    Collaborator

    Closing: the fix merged in #5872. If this still reproduces on a release that includes it, please reopen with the version and a repro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions