The Security guide explains the trust boundary. This page
lists the rules behind it, as the code enforces them.
Reference
The model reads app content as quoted evidence, kept apart from runner policy
and trusted project context. App content can’t grant tools, credentials, or a
larger budget. How agent steps work explains a step in full.
- Built-in tools have closed schemas and appear only when the engine
supports them.
- Project tools must use
defineTool with an execute and can’t replace a
built-in name.
- Only read-only project tools may request an observation.
- A call to a tool the step doesn’t offer, or with an undeclared argument,
never runs. The model reads the refusal as the call’s result and tries
again.
- The runner never evaluates model text as code, selectors, shell commands,
or config.
Policy denials
These calls fail with POLICY_DENIED.
Navigation
- The runner doesn’t re-check redirects or attach to popup pages. Register
browser.onDialog to handle browser dialogs.
There is no origin allowlist. The agent can follow the app to other sites
and use the step’s secrets there.
Site check
The browser engine compares hostnames to decide what counts as the target’s
site.
- Configured
headers go only on requests to the target’s site.
- Observations leave out child frames from other sites, and
data: frames.
- The check approximates the registrable domain without a public suffix
list. A shared hosting domain such as
vercel.app counts as one site, so
headers can reach other hosts under it.
basicAuth ignores the site check. It answers a 401 challenge from any
origin.
Secret fills
A Secret is an opaque handle with no plaintext accessor. Three sinks accept
one: a locator’s fill, the params of an agent step, and an engine option
that declares it.
The runner resolves the value on the host and passes it to the engine. The
model never sees or picks it.
Screenshots after a secret fill
After any secret fill, no pixels reach a model for the rest of the attempt:
- screenshot and pixel tools are unavailable,
- an
act or vision: true step whose semantic capture times out before the
step does fails with POLICY_DENIED.
The app could show the value anywhere, not only in the field’s masked
rectangle, and model input leaves your computer.
Screenshots the runner keeps as artifacts stay on: app.screenshot(),
toHaveScreenshot(), and assertion and failure screenshots.
An artifact screenshot masks secure fields only. If the app shows the value
anywhere else, the image has it. Treat screenshots like
videos.
A test that restores a session whose setup filled a secret keeps
this rule. A setup that signs in without a secret fill, for example by
setting a session cookie with browser.setCookies, leaves model pixels on for
the tests that restore it. A value it uses is redacted only if it’s a
configured secret.
Secrets in engine options
A secret in an engine option, such as
web({ basicAuth: { password: secrets.get(name) } }), isn’t a fill.
- The runner resolves it on the host when each attempt starts.
- It registers the value for redaction, together with what the engine
derives from it. For basic auth, that is the base64
user:password in the
Authorization header.
- It’s redacted from reports,
screen-at-failure.txt, trace pages, what the
model and an e2e mcp session read, and text downloads.
- Screenshots and the model’s pixels stay on. A page that shows the password
on screen isn’t masked in pixels.
Redaction
The runner replaces every registered secret value with <secret:name> in:
- model input,
- reports and trace pages,
- text downloads,
- what your code prints during
e2e run: tests, the top-level code of the
config and test files, and reporters.
It also redacts a value passed as a plain string instead of a
secrets.get() handle: test and describe titles, step labels (an
app.open URL, a locator’s text), a fill, an agent.act or agent.assert
instruction, the strings in agent.act params, agentContext, and an
e2e explore goal. A plain string isn’t a secret fill, so screenshots stay
on.
Where redaction stops
Redaction matches strings. These pass through:
- a transformed value, such as its last four characters,
- a value encoded as a whole, such as a hash, or base64 outside a text
download,
- a value in another Unicode normalization form, such as decomposed accents
where the secret has composed ones,
- text that drops the whitespace inside the value. Text that widens it passes
through too, except on the observed screen, where a field whose collapsed
form holds the value stays collapsed and masked,
- a value spread over several nodes, such as one character per cell on a PIN
pad,
- a case mapping that changes length (
ß shown as SS), for the cut-text
rule,
- a value a secret provider returns, in titles, since titles are redacted
before any provider runs,
- console output of tests that run in the runner’s own process, when an
embedding host passes the config or tests in memory,
- video: it masks nothing and is labeled
redaction: "incomplete". A
recording a hosted browser or device service keeps stays with that
service, and the report holds only its URL,
command.log: the runner captures your app’s output as written.
Downloads
A download is what the app served. When the session knows a secret value,
the runner rewrites a text download (.txt, .csv, .json, .html, or
.htm) and labels it redaction: "complete". It replaces with <secret:name>:
- every whole occurrence of a value,
- any run of 8 or more of its characters, whitespace collapsed or not, so
unrelated text that shares 8 characters with a value is masked too,
- a base64 or base64url run that decodes to either.
Every other download is kept as served and labeled
redaction: "incomplete":
- any other extension,
- one that isn’t valid UTF-8 or can’t be rewritten,
- any download from a session that knows no secret value.
Sessions
A saved session holds authentication state. It also carries, in the same
ciphertext, the secret values the setup test resolved from a provider. The
test that restores it redacts those values and keeps its pixels withheld. The
envelope holds, unencrypted but authenticated, the secret names and whether a
secret was filled.
The runner refuses an invalid, expired, missing, or mismatched session. See
the session errors.
Cache trust
Cache entries replay actions without a model and aren’t signed.
Artifacts
A run writes under .e2e/ by default. The
output option or --output moves it. The
replay cache stays at cache.dir, and command.log goes to the
project-relative path you configure.
An artifacts.store receives each artifact’s bytes as they’re produced,
after download redaction, with the redaction label the report records.
Outbound connections
There is no crash reporting.
--ai-trace writes a local file and doesn’t upload it. Your test code,
plugins, and app can make more connections.
MCP and explore