{{ message }}
feat: add audio probe#880
Open
thymikee wants to merge 2 commits into
Open
Conversation
Member
Author
|
Fallow Code Quality is failing on new complexity in the audio path. Please split/flatten these before review:
After updating, please rerun |
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an agent-device web audio probe for browser sessions:
audio probe start|status|stopsamples page media element audio into compact RMS/peak dBFS buckets that can be inspected while other actions continue.Wires the command through CLI, typed client, daemon routing, capabilities, batch validation, MCP metadata, and the managed agent-browser web provider. Also adds an Expo test-app Audio tab with a generated 8-second tracker-style
<audio>sample for dogfooding on web.CI follow-up: splits the audio request resolver and provider normalizer to satisfy Fallow, unwraps the real
agent-browser evalenvelope, and lets the probe rediscover media streams that appear afteraudio probe start.Tradeoff: this implementation samples HTML media elements exposed to Web Audio; it does not capture whole-tab, OS, or device audio.
Validation
Focused checks passed:
pnpm exec vitest run --project unit src/platforms/web/agent-browser-provider.test.ts,pnpm check:fallow --base 24cb2b622cf19f18f61bf646bd137fb1a819ee56,pnpm test-app:typecheck,pnpm check:quick,pnpm build, andpnpm format.Manual dogfood against the Expo web app used a live 20s probe while clicking the sample button:
agent-device audio probe start 20 1000 --platform web --session audio-ci-example --json agent-device click 'label="Start sample"' --platform web --session audio-ci-example --json agent-device audio probe status --platform web --session audio-ci-example --jsonReal status output excerpt from the run:
{ "audio": "probe", "state": "running", "heard": true, "durationMs": 20000, "elapsedMs": 8925, "bucketMs": 1000, "sampleCount": 9, "mediaElementCount": 1, "sourceCount": 1, "rmsDbfs": [-90, -90, -32, -33, -32, -33, -32, -33, -42], "peakDbfs": [-90, -90, -25, -26, -24, -26, -25, -26, -25] }