feat(cursor-sdk): Cursor SDK tracing instrumentor (TH-8263) by nik13 · Pull Request #253 · future-agi/traceAI · GitHub
Skip to content

feat(cursor-sdk): Cursor SDK tracing instrumentor (TH-8263) - #253

Open
nik13 wants to merge 7 commits into
devfrom
feat/th-8263-cursor-sdk
Open

nik13 wants to merge 7 commits into
devfrom
feat/th-8263-cursor-sdk

Conversation

@nik13

@nik13 nik13 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds @traceai/cursor-sdk (typescript/packages/traceai_cursor_sdk), an OpenTelemetry instrumentor for Cursor's official TypeScript SDK @cursor/sdk. Each agent.send() (and the run inside Agent.prompt()) becomes one trace:

Cursor Agent run              AGENT  session.id = agent id, model, tokens, status
├─ Cursor thinking            CHAIN  one per thinking block (deltas joined), duration
├─ Cursor assistant message   LLM    one per run of consecutive text deltas
└─ Cursor tool: <name>        TOOL   one per tool_call call_id: running -> completed | error
  • Patch points: the static Agent.create / Agent.resume (manuallyInstrument(CursorSDK) for ESM, the OpenTelemetry require hook for CJS). The agent they return gets a traced send; the Run it returns gets traced stream() / wait() and an onDidChangeStatus subscription. Agent.prompt() is traced through Agent.create -> send -> wait, which is how 1.0.37 implements it (checked against the installed SDK).
  • Run span: session.id (agent id, so runs of one agent form a session), gen_ai.conversation.id, cursor.runtime, run and request ids, model, status, duration, token usage (RunResult.usage / run.usage, else the sum of usage events; input, output, total, cache read/write, reasoning). ERROR on error / cancelled runs and on thrown errors.
  • Agent.create / Agent.resume failures (for example AuthenticationError for a wrong key) are one ERROR span Cursor Agent create / Cursor Agent resume.
  • No semantics change: callers get the SDK's own agent, run and event objects; errors are rethrown unchanged; the wrapper never reads the run stream itself (the local runtime's stream is a single-consumer queue).
  • Spans always end: stream end or throw, wait() settling, early break, a partly read stream never closed (streamDrainTimeoutMs, default 30 s, once the run's end is known), fire-and-forget (status listener plus a 2 s grace period), forceFlush() (runs whose end is known) and shutdown() (every open run).

SDK facts were read from cursor.com/docs/sdk/typescript and verified against the installed 1.0.37 .d.ts and dist/esm/index.js (text and thinking arrive as deltas, thinking-completed carries thinking_duration_ms, tool_call is emitted at start, on partial updates and at completion, usage once per turn end), then confirmed by live signed-in runs on 1.0.38 (below).

Live vendor verification (2026-10-20)

Signed-in Cursor account (Plus plan, grok_ user key), @cursor/sdk 1.0.38 (npm latest), @traceai/fi-core 1.0.0 from npm, this package from a pnpm pack tarball (the default run at a61dfcf; the content-on runs with the ed6ff22 code, before the example's cwd fix), Node 26.8, local runtime, composer-2.5 + fast: false, a throwaway git repo, fresh HOME, register({ setGlobalTracerProvider: false }). Four agent runs in total.

Run Trace Result
streamed, default privacy, project th8103-livefill-th8263 eec8abf8c9a94e3c86881f144403c24e run OK -> 3 thinking, read, edit (wrote notes.txt), 1 assistant message; 23820 input + 19172 cache read / 258 output, total 43250 (exported before the token fix, so stored prompt_tokens 23820)
examples/basic.mjs, content on via FI_HIDE_INPUTS=false / FI_HIDE_OUTPUTS=false, project th8103-livefill-th8263-content 581f15b8429948b7830e387361944f5a run OK -> 2 thinking, shell, read (parallel with the shell), shell, 1 assistant; stored prompt_tokens 43416 = 24165 + 19251 cached (after the fix), total 43734
same agent, wait() only 93c316ab674e4a59aa3c9f9ce9b14543 run OK, same session; 52352 / 524
composer-2.5 without params on Plus (real failure) 9529e2dfaba046829124a4b4b427fcf3 run ERROR RunError; Cursor's text "Model unavailable on Plus ... with fixed settings" not exported by default

fi_verify.py --project th8103-livefill-th8263 --wait 240 --min-spans 1 --expect-type tool: PASS (2 traces, 8 spans); same for the content project: PASS (2 traces, 8 spans). Only this package's span names in both projects; 0 context-tracer / LocalGitExecutor spans.

Privacy, /tmp/hr3/check_ch.py over every ClickHouse spans column: default project, prompt marker 0, README marker 0, /Users/ paths 0, tracebacks 0, status messages only RunError; content project, prompt in input (1 row), README marker in outputs (3), home paths in tool args/results (3 + 1). The API key (and its first 12 characters) 0 in both.

Differences from the replay, fixed in ed6ff22 / 2a65450 with tests:

  • Tokens. Cursor's totalTokens = input + output + cache read + cache write; inputTokens excludes the cache. The replay assumed total = input + output. gen_ai.usage.input_tokens now adds cache read/write (GenAI semconv), so input + output = total; cursor.usage.* keeps Cursor's numbers.
  • Model params. { id: "composer-2.5" } fails on Plus ("Model unavailable on Plus ... fixed settings"); README, example and docs pass params: [{ id: "fast", value: "false" }] from Cursor.models.list() and troubleshoot the RunError.
  • Relative local.cwd breaks the agent's shell tool ("working directory no longer exists"); the example now resolves it.
  • No request / user / system / task events in a local run (request id only on RunResult.requestId, which was already read); RunResult.error can have no code (status message RunError); tool ids tool_<uuid>, no truncated, parallel tool calls; a non-zero shell exit is a completed call. All replayed in new jest cases from the recorded sequences.
  • agent.getUsage() returned 403 feature_unavailable on Plus; README/docs no longer send users there unconditionally.
  • Dashboard moved to grok.com/dashboard; links updated. Dev dependency @cursor/sdk 1.0.38 (lockfile: only the @cursor/sdk* entries and its zod peer).

Matched the replay: thinking deltas closed by an empty-text event with thinking_duration_ms; tool_call running (repeated for partial updates) then completed; one usage per turn; status RUNNING / FINISHED / ERROR.

Review round (2026-10-15)

  • Error text is content. Unless both hideInputs and hideOutputs are false, an error exports only exception.type (no exception.message, no exception.stacktrace), the structural cursor.error.name/code/http_status/retryable/operation, and a fixed status message / cursor.error.message <ErrorName> (<http status or code>), e.g. AuthenticationError (401); failed runs RunError (<code>). RunResult.error and status-event text only on opt-in. Previously a wrong key exported the SDK's stack (checkout path with the username) and raw messages.
  • SDK-internal spans. @cursor/sdk bundles @opentelemetry/api and starts its own spans (scope context-tracer) on the global provider: 27 in one missing-key local run. README, example and docs now use register({ ..., setGlobalTracerProvider: false }) with the provider passed explicitly; the contract asserts the exporter then gets 0 of them.
  • Partly read stream leak fixed; unpatch() now also stops agents created earlier; README "verified" line made exact, "Not captured / limitations" section, wrong-key vs missing-key troubleshooting, onStep/onDelta noted as unused by this version.
  • The real-SDK Agent.resume contract check (dropped in the first rewrite) is back with the redaction assertions (a61dfcf).
  • The Codex SDK instrumentor (feat(codex-sdk): OpenAI Codex SDK tracing instrumentor (TH-8262) #251) has the same unconditional recordException(error); not changed here.

Why?

TH-8263 (TH-8103 coding-agent family): trace Cursor SDK agent runs in Future AGI. Sibling packages: @traceai/codex-sdk (TH-8262), @traceai/pi-coding-agent (TH-8264).

How was it tested?

  • Unit tests: 48 jest cases (4 replay the live-recorded 1.0.38 sequences) (pnpm test) on a fake SDK with the real 1.0.37 shapes and control flow: every documented event, content hidden / opt-in, error text redaction scanning attributes, span events and status messages with an error carrying a home-directory path and username (absent by default, present on opt-in, still redacted when only one side is opted in), tool error, failed and cancelled runs, stream throw, send() / create() / resume() / wait() rejecting, wait-only, Agent.prompt(), early break with and without status listeners, partly read stream, fire-and-forget, late stream start, concurrent stream + wait, forceFlush() / shutdown(), sessions, patch / unpatch / disable, malformed events.
  • Contract (HOME=$(mktemp -d) pnpm run test:contract), REAL @cursor/sdk 1.0.37, no account: type check of the real SDK types (contract/sdk_types.ts); CJS auto-instrumentation; wrong-key Agent.create / Agent.prompt and unknown-agent Agent.resume (identical error to the caller; by default no message, stack, checkout path or file:// in attributes, events or status, status message AuthenticationError (401) / AgentNotFoundError (agent_not_found); full message and stack on opt-in); a missing-key local run through the real runtime's send() / stream() / wait() (events status:RUNNING, status:ERROR, run span ERROR RunError, prompt and paths absent) where the explicitly passed, non-global provider receives only @traceai/cursor-sdk spans. 6/6 contract checks passed against @cursor/sdk 1.0.37, and again on 1.0.38 at 2a65450.
  • tsc strict build (CJS, ESM, ESNext). The TS workspace has no ESLint config, so the strict build is the lint.
  • Install from source as the docs page describes (shallow clone of this branch from GitHub, pnpm install --filter "@traceai/cursor-sdk...", build, pnpm pack, npm install of the tarball with npm's @traceai/fi-core 1.0.0 and @cursor/sdk): installs and imports in a fresh app.
  • E2E into a local Future AGI through register({ setGlobalTracerProvider: false }) (below), and the live signed-in runs above.

Replay E2E evidence (before the live runs; local Future AGI, project th8103-coding-cursor-sdk, re-run 2026-10-15 at a108ae5; a61dfcf only adds a contract check)

Ran: (a) a fake agent emitting the documented 1.0.37 events (streamed run, wait-only follow-up, and a failing run whose Cursor error text quotes a home-directory path) with default privacy; (b) the REAL SDK local runtime with no key (send/stream/wait, Cursor rejects the run); (c) the REAL SDK with a wrong key (Agent.create, Agent.prompt); (d) (a) again with content opted in. All through register({ setGlobalTracerProvider: false }) with the provider passed explicitly. fi_verify.py ... --expect-type tool: PASS.

4555c51c-fbba-41b2-8eaf-50a3f64e31f7  7 spans  Cursor Agent run OK     tok 4211/187  run -> thinking, 2 assistant, ls, read, read (ERROR)
d64ba51c-c439-4d73-833c-9ce958da2be0  1 span   Cursor Agent run OK     (wait-only, same session b6f87e1a-...)
b8d1037f-98a3-4f50-8654-c60f46d894e8  2 spans  Cursor Agent run ERROR "RunError (internal)" -> Cursor tool: shell ERROR "shell failed" (same session)
6075ec68-14ea-4f58-9398-4e1dfcff14a3  1 span   Cursor Agent run ERROR "RunError"                        <- real SDK local runtime, no key
b5cbcf10-92e3-4a54-80e0-9f699fecfdb3  1 span   Cursor Agent create ERROR "AuthenticationError (401)"   <- real SDK, wrong key
73bacac3-b290-4918-8fd1-1822bad73e29  1 span   Cursor Agent create ERROR "AuthenticationError (401)"   <- real SDK, Agent.prompt
2ba4d46d-... / 627dc2f0-... / fccbde64-...                                                               <- content opted in
PASS

No span other than this package's ones landed in the project from the real local run (span names since the re-run: Cursor Agent create, Cursor Agent run, Cursor assistant message, Cursor thinking, Cursor tool: ls|read|shell), although the SDK starts about 27 internal context-tracer spans per local run on the global provider.

Privacy

Prompts, tool arguments, tool results, thinking, assistant text, task summaries and error text are hidden by default (same default as the Codex SDK and Pi SDK packages; traceConfig: { hideInputs: false, hideOutputs: false } or FI_HIDE_INPUTS=false / FI_HIDE_OUTPUTS=false opt in; error text needs both). The API key is never read. ClickHouse grep over the re-run's 23 spans (status_message, span_events, input, output and every string attribute): the 6 default-config traces contain the run marker 0 times, home-directory paths 0, stacktrace 0, the raw Invalid User API Key text 0; their status messages are only AuthenticationError (401), RunError, RunError (internal), read failed, shell failed. The 3 opted-in traces contain the marker 14 times and the quoted paths 5 times. API key strings 0. Span events are checked at the exporter in jest and in the contract (no exception.message / exception.stacktrace by default).

Not available / limitations

  • Tool, thinking and assistant spans need the app to read run.stream(); wait()-only and Agent.prompt() give the run span only. This version does not use the SDK's onStep / onDelta callbacks (possible follow-up: chain them in the wrapped send()).
  • Tokens per run (the SDK reports per turn); no per-model-call spans; no cost (use agent.getUsage()).
  • Runs not started through a traced send() (Agent.getRun(), Agent.listRuns()) are not traced.
  • Cloud runtime: same span model, not run end to end (the live runs were local).
  • A shell command that exits non-zero is a completed tool call (span OK); the exit code is only in the result with content on.
  • SDK-internal context-tracer spans are not filtered by this package; the docs keep the FI provider non-global.
  • pnpm-lock.yaml also carries pnpm's re-resolution churn of a few @types/node peer suffixes in other importers (same as the Codex SDK PR).

BLOCKED-ON

Resolved on 2026-10-20 for the local runtime (live vendor verification above). Still unproven: a cloud (bc-) agent run end to end, which needs a GitHub repo for Cloud Agents; not attempted.

Checklist

  • Branch is off dev
  • Conventional commits
  • No TODOs or commented-out code
  • No real API keys or secrets in the diff (contract uses a fake crsr_th8263_contract_invalid_key_do_not_use)
  • README documents the API, spans, privacy (including error data), limitations

🤖 Generated with Claude Code

nik13 and others added 2 commits October 14, 2026 19:06
@traceai/cursor-sdk traces @cursor/sdk (Cursor's TypeScript SDK) agent
runs. It patches the static Agent.create and Agent.resume (manuallyInstrument
for ESM, the OpenTelemetry require hook for CJS); the returned agent's send()
opens one AGENT span per run ("Cursor Agent run", session.id = agent id, so
the runs of one agent form a session) and the returned Run's stream() and
wait() feed it. Agent.prompt() is traced through Agent.create/send/wait,
which is how the SDK implements it.

Child spans: one TOOL span per tool_call call_id (running -> completed or
error, ERROR on error), one CHAIN span per thinking block and one LLM span
per run of assistant text deltas (the local runtime streams both as deltas).
The run span carries model, run/request ids, status (ERROR on error and
cancel, with Cursor's message), duration and token usage (RunResult/run
usage, else the sum of usage events; cache and reasoning tokens included).
Agent.create/resume failures such as AuthenticationError are one ERROR span
with the exception and the SDK error fields.

The SDK's own agent, run and event objects are returned unchanged; the
wrapper never reads the single-consumer stream itself. Spans end on early
break (tools marked incomplete, the run closes via onDidChangeStatus),
wait()-only use, fire-and-forget (status listener plus a short grace period),
and thrown errors. Prompts, tool arguments/results, thinking and assistant
text are hidden by default, like the Codex and Pi SDK packages
(FI_HIDE_INPUTS/FI_HIDE_OUTPUTS=false or traceConfig opt in); the API key is
never read.

Tests: 37 jest cases on a fake SDK with the real 1.0.37 shapes and control
flow, a shape guard on the installed SDK, a compile-time check of the real
SDK types (contract/sdk_types.ts), a CJS auto-instrumentation check, and a
real-SDK run with an invalid API key (AuthenticationError recorded, caller
sees the identical error, key absent from spans).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…TH-8263)

A bad CURSOR_API_KEY made the example exit before shutdown(), so the
Cursor Agent create ERROR span could be lost. Create the agent inside the
try so finally always flushes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 and others added 3 commits October 15, 2026 04:26
… (TH-8263)

Review round for #253.

Error text is content. Unless both hideInputs and hideOutputs are false,
an error now exports only its class and structural fields: the exception
event carries exception.type (no exception.message, no stacktrace), and
the status message and cursor.error.message are a fixed
"<ErrorName> (<http status or code>)", e.g. "AuthenticationError (401)".
Failed runs use "RunError (<code>)"; Cursor's RunResult.error and status
event text are exported only on opt-in. Before this, a bad key exported
the SDK's stack (local checkout path, username) and raw messages.

A stream read partly and never closed no longer holds the run span open:
once the run's end is known the span closes after streamDrainTimeoutMs
(default 30 s), open tools marked incomplete. forceFlush() closes every
run whose end is known; shutdown() closes every open run.

unpatch() now also stops agents created before it, as documented, until
manuallyInstrument() or wrapAgent() is called again.

Tests: privacy tests scan span events and status messages with an error
carrying /Users/<name>/ (absent by default, present on opt-in); partial
stream, forceFlush, shutdown and unpatch cases (44 jest). The real-SDK
contract now checks no message, stack or checkout path by default, the
full stack on opt-in, and a missing-key local run through the real
runtime's send/stream/wait where the explicitly passed, non-global
provider receives 0 SDK-internal spans (the SDK emitted 27
"context-tracer" spans to the global provider in that run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…H-8263)

README and examples/basic.mjs register the Future AGI provider with
setGlobalTracerProvider: false and pass it explicitly, so the SDK's own
internal OpenTelemetry spans (scope "context-tracer") never reach Future
AGI. The README now says exactly what was verified (real 1.0.37 error
paths and a missing-key local run; event mapping by replaying the
documented events; live signed-in run pending), adds a "Not captured /
limitations" section, documents what error data is exported, splits the
wrong-key and missing-key troubleshooting cases, words the stream-only
child spans as a limitation of this version (onStep/onDelta unused), and
uses a try/finally that always flushes. Example model: composer-2.5 (any
id from Cursor.models.list()).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(TH-8263)

The review-round rewrite of contract/real_sdk_error.mjs dropped the
real-SDK Agent.resume check that the README and the docs page cite. It is
back, with the redaction assertions: resuming an unknown local agent gives
the caller the identical AgentNotFoundError, one "Cursor Agent resume"
ERROR span with session.id = the agent id, the fixed status message
"AgentNotFoundError (agent_not_found)" and only exception.type; the raw
message (it quotes the agent id), stack trace and checkout path are not
exported. 6/6 contract checks pass against @cursor/sdk 1.0.37.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nik13 and others added 2 commits October 20, 2026 02:54
….38 runs (TH-8263)

A live run on @cursor/sdk 1.0.38 showed that Cursor's totalTokens is
input + output + cache read + cache write and its inputTokens excludes the
cache (23820 + 258 + 19172 = 43250). gen_ai.usage.input_tokens now adds the
cache tokens, as the GenAI semantic conventions ask, so input + output =
total; cursor.usage.* keeps Cursor's numbers.

Tests replay event sequences recorded from the live runs: no request /
user / system / task events (request id from RunResult.requestId), thinking
blocks closed by an empty-text duration event, repeated running updates,
parallel tool calls, a non-zero shell exit as a completed call, and a
plan-rejected run whose RunResult.error has no code (RunError). The fixture
usage now follows Cursor's total. Dev dependency @cursor/sdk 1.0.38 (the
real-SDK contract passes 6/6 on it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d (TH-8263)

README gains a "Live vendor verification (2026-10-20)" section (Plus plan,
@cursor/sdk 1.0.38, composer-2.5, trace ids, privacy result) and drops the
pending-live-run caveats. The example and README pass the model's params
(Plus rejects composer-2.5 without fast=false: "Model unavailable on Plus")
and the example resolves local.cwd (a relative cwd breaks the shell tool).
Troubleshooting covers both, the dashboard is grok.com/dashboard, the token
section explains Cursor's cache accounting, and agent.getUsage() returned
403 feature_unavailable on Plus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nik13
nik13 marked this pull request as ready for review October 21, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant