docs(skill): verified gotchas for headless default, frames, scrolling, tabs, emulation by TheLordWizard · Pull Request #488 · microsoft/playwright-cli · GitHub
Skip to content

docs(skill): verified gotchas for headless default, frames, scrolling, tabs, emulation - #488

Closed
Clinton Monteiro (TheLordWizard) wants to merge 1 commit into
microsoft:mainfrom
TheLordWizard:skill-verified-gotchas
Closed

Clinton Monteiro (TheLordWizard) wants to merge 1 commit into
microsoft:mainfrom
TheLordWizard:skill-verified-gotchas

Conversation

@TheLordWizard

Copy link
Copy Markdown

Why

We run this CLI inside a coding-agent harness (the model reads the skill file and drives the CLI
literally). Over a session on 2026-10-22 we hit a set of behaviours that the skill does not mention, each
of which cost a step or nearly produced a wrong claim. Everything below is measured on 0.1.22 (a few
on 0.1.20, labelled as history), on Windows, and the section is written as documentation, not as a
complaint. The patch is additive: +146 lines, no deletions.

What the patch adds

section the measurement behind it
Watching what the agent did the default launch carries --headless in the browser process command line; --headed leaves zero such processes; open --help lists only --browser/--config although --headed/--persistent/--profile work; show screencasts headless sessions too; video-start --cursor/--fps/--size — and --cursor paces actions by ~800 ms, which is worth knowing before a timing-sensitive run
Bot walls Google …/sorry/index after a handful of requests while the same session's normal search stayed clean; an automated reCAPTCHA click is refused silently (aria-checked:false, no token, no challenge frame); one human solve in a --persistent --profile survives close+reopen; “an empty selector is not a wall” (Google News headlines are a.JtKRv, there are no <article> elements — 217 vs 0)
Iframes a YouTube video stayed at currentTime 0.76 through two clicks because the consent modal's button was inside a frame; iterating page.frames() fixes it; frameLocator for the CAPTCHA checkbox
Verify the action, not the exit code a 7-row action → state-read table (URL/title, aria-checked, page verdict text, scrollY, start=10, player state 1, matchMedia(...).matches)
Scrolling on Reddit scroll-down, mouse-wheel and PageDown all left scrollY at 0 while the page was scrollable (scrollHeight 3783 vs innerHeight 1277, window.scrollTo(0,900) moved it); locator.scrollIntoViewIfNeeded() is the reliable route
Locator misses strict mode violation → .first()/.nth(n); a role/name miss → probe hrefs (slugs differ from titles); find --regex=… --filename=f keeps matches out of the response (391 ms, 15 KB to file); there is no --limit
One JS call instead of a chain 3 tasks × 2 reps × 2 arms, each graded against an independent page-state read: −75% calls, −42/−43% wall, identical correctness (12/12); context bytes depend on --raw discipline (bulk ~20% smaller than realistic per-call output, ~28% larger than a --raw chain), so the claim is turns, not tokens
Timeouts, pacing, tabs the shipped defaults {action:5e3, navigation:6e4, expect:5e3, settle:500} (read out of the installed bundle); --idle-timeout does not apply to a headed session; outbound links open a new tab and focus stays behind, so a click that changes no Page URL usually succeeded; there is no --new-tab flag (0.1.20 silent no-op, 0.1.22 prints usage and still opens nothing — #486)

Not in this patch

Nothing unmeasured, and nothing machine-specific (no local paths, no tuning).

One thing we use that may interest you separately: the six gates above are driven by a small contract
suite for the skill itself — an offline fixture (pointer drag, computed-colour reading, a 3 s press-and-hold,
a click inside an iframe, an answer that exists only in page state, element-based scroll visibility) plus a
15-case runner that drives the real CLI and asserts on page/process state instead of exit codes
(SKILLCHECK cases=15 pass=15, exit 1 on failure; it serves the fixture over 127.0.0.1 because file:
is blocked). If you want it as a CI tool, say so and I'll send it as a second commit — I left it out here
because adding test infrastructure to someone's repo is a bigger ask than documenting what we measured.

Refs: #486 (the --new-tab no-op, with a 0.1.22 follow-up comment), #487 (clear-* does not revert emulation
— that bug is in the same family: a command that succeeds and changes no state).

…, tabs, emulation

Signed-off-by: TheLordWizard <TheLordWizard@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants