Security rules - e2e
Skip to main content
The Security guide explains the trust boundary. This page lists the rules behind it, as the code enforces them.

Reference

Model input

The model reads app content as quoted evidence, kept apart from runner policy and trusted project context. App content can’t grant tools, credentials, or a larger budget. How agent steps work explains a step in full.

Tool calls

  • Built-in tools have closed schemas and appear only when the engine supports them.
  • Project tools must use defineTool with an execute and can’t replace a built-in name.
  • Only read-only project tools may request an observation.
  • A call to a tool the step doesn’t offer, or with an undeclared argument, never runs. The model reads the refusal as the call’s result and tries again.
  • The runner never evaluates model text as code, selectors, shell commands, or config.

Policy denials

These calls fail with POLICY_DENIED.
  • The runner doesn’t re-check redirects or attach to popup pages. Register browser.onDialog to handle browser dialogs.
There is no origin allowlist. The agent can follow the app to other sites and use the step’s secrets there.

Site check

The browser engine compares hostnames to decide what counts as the target’s site.
  • Configured headers go only on requests to the target’s site.
  • Observations leave out child frames from other sites, and data: frames.
  • The check approximates the registrable domain without a public suffix list. A shared hosting domain such as vercel.app counts as one site, so headers can reach other hosts under it.
  • basicAuth ignores the site check. It answers a 401 challenge from any origin.

Secret fills

A Secret is an opaque handle with no plaintext accessor. Three sinks accept one: a locator’s fill, the params of an agent step, and an engine option that declares it.
tests/stripe.e2e.ts
The runner resolves the value on the host and passes it to the engine. The model never sees or picks it.

Screenshots after a secret fill

After any secret fill, no pixels reach a model for the rest of the attempt:
  • screenshot and pixel tools are unavailable,
  • an act or vision: true step whose semantic capture times out before the step does fails with POLICY_DENIED.
The app could show the value anywhere, not only in the field’s masked rectangle, and model input leaves your computer. Screenshots the runner keeps as artifacts stay on: app.screenshot(), toHaveScreenshot(), and assertion and failure screenshots.
An artifact screenshot masks secure fields only. If the app shows the value anywhere else, the image has it. Treat screenshots like videos.
A test that restores a session whose setup filled a secret keeps this rule. A setup that signs in without a secret fill, for example by setting a session cookie with browser.setCookies, leaves model pixels on for the tests that restore it. A value it uses is redacted only if it’s a configured secret.

Secrets in engine options

A secret in an engine option, such as web({ basicAuth: { password: secrets.get(name) } }), isn’t a fill.
e2e.config.ts
  • The runner resolves it on the host when each attempt starts.
  • It registers the value for redaction, together with what the engine derives from it. For basic auth, that is the base64 user:password in the Authorization header.
  • It’s redacted from reports, screen-at-failure.txt, trace pages, what the model and an e2e mcp session read, and text downloads.
  • Screenshots and the model’s pixels stay on. A page that shows the password on screen isn’t masked in pixels.

Redaction

The runner replaces every registered secret value with <secret:name> in:
  • model input,
  • reports and trace pages,
  • text downloads,
  • what your code prints during e2e run: tests, the top-level code of the config and test files, and reporters.
It also redacts a value passed as a plain string instead of a secrets.get() handle: test and describe titles, step labels (an app.open URL, a locator’s text), a fill, an agent.act or agent.assert instruction, the strings in agent.act params, agentContext, and an e2e explore goal. A plain string isn’t a secret fill, so screenshots stay on.

Where redaction stops

Redaction matches strings. These pass through:
  • a transformed value, such as its last four characters,
  • a value encoded as a whole, such as a hash, or base64 outside a text download,
  • a value in another Unicode normalization form, such as decomposed accents where the secret has composed ones,
  • text that drops the whitespace inside the value. Text that widens it passes through too, except on the observed screen, where a field whose collapsed form holds the value stays collapsed and masked,
  • a value spread over several nodes, such as one character per cell on a PIN pad,
  • a case mapping that changes length (ß shown as SS), for the cut-text rule,
  • a value a secret provider returns, in titles, since titles are redacted before any provider runs,
  • console output of tests that run in the runner’s own process, when an embedding host passes the config or tests in memory,
  • video: it masks nothing and is labeled redaction: "incomplete". A recording a hosted browser or device service keeps stays with that service, and the report holds only its URL,
  • command.log: the runner captures your app’s output as written.

Downloads

A download is what the app served. When the session knows a secret value, the runner rewrites a text download (.txt, .csv, .json, .html, or .htm) and labels it redaction: "complete". It replaces with <secret:name>:
  • every whole occurrence of a value,
  • any run of 8 or more of its characters, whitespace collapsed or not, so unrelated text that shares 8 characters with a value is masked too,
  • a base64 or base64url run that decodes to either.
Every other download is kept as served and labeled redaction: "incomplete":
  • any other extension,
  • one that isn’t valid UTF-8 or can’t be rewritten,
  • any download from a session that knows no secret value.

Sessions

A saved session holds authentication state. It also carries, in the same ciphertext, the secret values the setup test resolved from a provider. The test that restores it redacts those values and keeps its pixels withheld. The envelope holds, unencrypted but authenticated, the secret names and whether a secret was filled. The runner refuses an invalid, expired, missing, or mismatched session. See the session errors.

Cache trust

Cache entries replay actions without a model and aren’t signed.

Artifacts

A run writes under .e2e/ by default. The output option or --output moves it. The replay cache stays at cache.dir, and command.log goes to the project-relative path you configure. An artifacts.store receives each artifact’s bytes as they’re produced, after download redaction, with the redaction label the report records.

Outbound connections

There is no crash reporting. --ai-trace writes a local file and doesn’t upload it. Your test code, plugins, and app can make more connections.

MCP and explore