You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d01c072
Browse filesBrowse the repository at this point in the historyBrowse files
agentHost: Measure discovery result processing and registration (microsoft#338791)
* agentHost: Measure discovery result processing and registration
Distinguish provider enumeration from processed discovery results and bounded host registration batches. Preserve startup ordering, telemetry consent, and scan semantics while documenting the publication boundary and adding deterministic coverage.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test: Assert successful Codex discovery completion
Supply the startup telemetry hook in the prototype harness and check the discovery result so swallowed exceptions cannot leave the classification test green.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@@ -66,11 +66,11 @@ start. Pair these markers by **host ID, provider, and attempt**, not just name.
66
66
The handle also protects against overlapping requests, stale completions, and
67
67
unbounded retries; callers do not manage timestamps or correlation tokens.
68
68
69
-
Keep workload aggregation and payload construction in focused helpers, invoked
70
-
only for an observed operation or a pending first-list milestone with collection
71
-
enabled. This keeps the measured control flow readable and skips telemetry-only
72
-
counting afterward. Helpers reuse existing snapshots and results, rather than
73
-
repeating enumeration or database work.
69
+
Keep operation bookkeeping, workload aggregation and payload construction in
70
+
focused helpers, leaving the measured control flow to name boundaries and
71
+
outcomes. Aggregate only for an observed operation or a pending first milestone
72
+
with collection enabled; skip telemetry-only counting afterward. Helpers reuse
73
+
existing snapshots and results, rather than repeating enumeration or database work.
74
74
75
75
## Boundaries
76
76
@@ -90,9 +90,12 @@ repeating enumeration or database work.
90
90
|`sessionMigration`| Provider catalog import, including enumeration when needed, synchronization, and publication. |
91
91
|`sessionMigrationScan`| Provider enumeration for migration, before filtering to migratable sessions. |
92
92
|`sessionDiscoveryScan`| Provider enumeration for external-chat discovery, before visibility/subagent filtering. |
93
+
|`firstSessionDiscoveryResult`| Node performance time origin until the first provider discovery pass finishes processing its result and synchronously emitting its candidate batches. Also records a processed zero-candidate result when no event is emitted. Does not wait for host registration or imply exhaustive/error-free enumeration. |
94
+
|`sessionDiscoveryRegistration`| One received discovery batch's host registration processing: registry snapshots, host filtering, accepted registry writes, catalog synchronization, and existing post-processing. Starts when the batch reaches registration, after its queue wait and any forced migration. Ends after scheduling, **not awaiting**, external-session publication reconciliation. |
95
+
|`firstSessionDiscoveryRegistration`| Node performance time origin until the first discovery registration batch completes without an observed registration/post-processing failure or pending catalog synchronization. Includes that batch's counts, independently of operation sampling. Not completion of all batches or all providers. |
93
96
|`sessionMetadataScan`| Copilot bulk enumeration used to prewarm metadata on catalog fallback. |
94
97
95
-
The five`session*` operations also emit corresponding `*Start` markers.
98
+
The six`session*` operations also emit corresponding `*Start` markers.
96
99
`bootstrap`, `hostReady`, `firstSessionList`, and `startupSettled` are cumulative
97
100
milestones, not additive phases. The latter two are independent of the operation
98
101
retry cap. They have no `outcome` field; a failed listing cannot emit them.
@@ -104,6 +107,13 @@ until explicit use. An observed scan is not necessarily on the startup critical
104
107
path: use its start marker and the host milestones to distinguish those cases.
105
108
Collection never activates or downloads a provider solely to obtain counts.
106
109
110
+
The two `firstSessionDiscovery*` milestones are also cumulative, once per
111
+
provider, and have no `outcome` field. A provider result can contain only a
112
+
subset of its catalog (for example, a truncated Codex scan or Copilot
113
+
classification failures). A successful registration is about the received
114
+
batch, not the completeness of its source scan. Neither milestone means that
115
+
an external row has been published to clients or rendered.
Coalesced session-list reconciliation and deferred title work run separately
231
257
```
232
258
259
+
Provider processing and host registration can overlap: Copilot emits multiple
260
+
batches during one pass, so the first registration can finish before
261
+
`firstSessionDiscoveryResult`. Conversely, a provider can finish emitting all
262
+
batches before registration starts. Codex can share an enumeration with
263
+
migration, so a discovery result need not have a `sessionDiscoveryScan` marker.
264
+
Do not pair scans and registration batches by attempt, proximity, or the latest
265
+
successful scan. Only pair each operation with its own `*Start` within the same
266
+
host, provider, and attempt.
267
+
268
+
The result milestone observes the first normally completed processing pass,
269
+
including zero candidates after filtering or deduplication. Copilot and Codex
270
+
do not emit empty candidate events; their result milestone can therefore exist
271
+
without a registration operation. Claude's existing empty event still takes
272
+
the existing host path. A deferred/unavailable provider, thrown processing
273
+
failure, or shutdown-abandoned result does not record this milestone. Later
274
+
readiness/discovery can record it, even after scan sampling stops. Provider
275
+
restarts do not reset either first-discovery milestone within the host lifetime.
276
+
233
277
Migration and discovery scans exist for Copilot, Claude and Codex; metadata
234
278
prewarming is Copilot-only. Codex scan end markers also include `pageCount` and
235
279
`truncated`. Legacy migration or catalog backfill can start discovery before
@@ -249,6 +293,8 @@ not every startup produces every marker.
249
293
|`sessionListStart`, `sessionList`| Around `_computeSessions` in [the agent service](node/agentService.ts). |
250
294
|`sessionMigrationStart`, `sessionMigration`| Around provider catalog import in [the agent service](node/agentService.ts). |
251
295
|`sessionMigrationScanStart`, `sessionMigrationScan`, `sessionDiscoveryScanStart`, `sessionDiscoveryScan`| Around enumeration in [Copilot](node/copilot/copilotAgent.ts), [Claude](node/claude/claudeAgent.ts) and [Codex](node/codex/codexAgent.ts). |
296
+
|`firstSessionDiscoveryResult`| After result processing and candidate emission in [Copilot](node/copilot/copilotAgent.ts), [Claude](node/claude/claudeAgent.ts) and [Codex](node/codex/codexAgent.ts), including normally completed empty results. |
297
+
|`sessionDiscoveryRegistrationStart`, `sessionDiscoveryRegistration`, `firstSessionDiscoveryRegistration`| In `_registerDiscoveredChatsWithStartupTelemetry`, wrapping registration after the existing queued migration in [the agent service](node/agentService.ts). |
252
298
|`sessionMetadataScanStart`, `sessionMetadataScan`| In the SDK enumeration wrapper used by `prewarmSessionMetadata` in [Copilot](node/copilot/copilotAgent.ts). |
253
299
254
300
## Workload counts
@@ -280,6 +326,59 @@ Migration records include synchronized, already-current, excluded, incomplete,
280
326
and failed candidate counts. `incompleteSessionCount` includes stale exclusions.
281
327
A deferred or unavailable catalog has no scan count, not a synthetic zero.
282
328
329
+
### Discovery counts
330
+
331
+
`firstSessionDiscoveryResult` records `candidateSessionCount` and
332
+
`externalSessionCount`: candidates actually emitted by that processing pass,
333
+
and the subset the provider classified as external. Copilot aggregates all
334
+
of its emitted batches after deduplication. Claude counts its unknown chats;
335
+
Codex counts new/changed chats, which can include known internal chats.
336
+
These are not catalog totals or host registrations.
337
+
338
+
Copilot also records `filteredSessionCount` (enumerated entries deliberately
339
+
not emitted, including known, suppressed, stale and duplicate entries) and
340
+
`failedSessionCount` (entries skipped after a caught classification failure).
341
+
Claude records its known-chat filtering count. Codex omits filtering counts
342
+
because native-subagent filtering happens inside the shared listing path;
343
+
Claude and Codex omit failure counts rather than claiming their lower-level
344
+
metadata recovery paths were error-free. Missing counts are unknown, not zero.
345
+
A zero-candidate result only describes the observed result, not an empty
346
+
provider-wide catalog. In particular, retain Copilot's failed count and Codex's
347
+
separately observed scan truncation when interpreting coverage.
348
+
349
+
`sessionDiscoveryRegistration` and `firstSessionDiscoveryRegistration` use
350
+
counts from **one received batch**, not accumulated provider totals:
351
+
352
+
-`candidateSessionCount`: entries in that event.
353
+
-`externalSessionCount`: entries the provider marked external, before host
354
+
provenance corrections and visibility filtering.
355
+
-`registeredSessionCount`: accepted registry writes, even if later processing
356
+
of that entry fails. This does not count published/visible rows.
357
+
-`filteredSessionCount`: entries deliberately skipped without a new
358
+
registration (already registered, subagent/backing, stale, or tombstoned).
359
+
-`failedSessionCount`: entries whose processing threw into the registration
360
+
loop's existing per-candidate error handler.
361
+
-`incompleteSessionCount`: entries whose catalog synchronization returned
362
+
`pending`.
363
+
364
+
Counts need not partition the batch: an accepted write can precede a failure
365
+
or pending synchronization. Batch-level errors do not invent per-candidate
366
+
failures, and cancellation only retains progress actually collected.
367
+
An observed per-entry failure, pending synchronization, or caught
368
+
post-processing write failure yields `partial`, not `success`, even though
369
+
product error handling still logs and continues. An escaping batch error
370
+
yields `error`; cancellation or a no-longer-current provider yields `cancelled`.
371
+
Recovered failures internal to metadata helpers are not an exhaustive error
372
+
count. Publication reconciliation can later fail independently of a successful
373
+
registration observation.
374
+
375
+
For first registration latency, use
376
+
`firstSessionDiscoveryRegistration.durationMs`, grouped by provider. It includes
377
+
the time before that batch, unlike the sampled registration operation duration.
378
+
Keep result-only, partial, missing and late observations separate; do not
379
+
manufacture an all-providers-complete timestamp or label either boundary
380
+
"visible". The first successful batch may itself be empty or entirely filtered.
381
+
283
382
For a graph of listing latency by Copilot volume, filter successful
284
383
`sessionList` records and group `durationMs` by `copilotSessionCount < 100`,
285
384
`== 100`, and `> 100`. Use `firstSessionList.durationMs` for process-relative
@@ -343,11 +442,14 @@ double-counted. Unknown providers collapse to `other` before keys or telemetry
343
442
are retained. Each attempt emits at most two markers; milestones and provider
344
443
context snapshots emit once per key. There are no per-session events, timers, performance observers,
345
444
additional filesystem reads, SDK calls, or telemetry-triggered background work.
346
-
The only additional session iteration counts an already-loaded registry snapshot
347
-
for an observed listing or a computation competing to be the first successful
348
-
listing. Metrics are not aggregated when collection is disabled. Later
349
-
listings/scans do not allocate timing handles or read the clock once their
350
-
observation and the first-list milestone are complete.
445
+
Additional counting uses only already-loaded registry snapshots and discovery
446
+
candidate arrays, for an observed operation or a pending first milestone.
447
+
`isPending` checks the same normalized milestone keys without reading a clock;
448
+
combine it with `isEnabled` before telemetry-only aggregation. Disabled
449
+
completions still consume their milestones without payload aggregation, so they
450
+
are not replayed after opt-in. Later listings/scans/registrations do not allocate
451
+
timing handles or read the clock once their observations and first milestones
452
+
are complete.
351
453
352
454
Markers use the existing telemetry transport without awaiting a flush
353
455
on the startup path. Telemetry sink exceptions are logged and cannot fail the
@@ -360,7 +462,9 @@ Outcomes are `success`, `error`, `unavailable`, `deferred`, `partial`, and
360
462
`cancelled`. Open observations are cancelled on runtime disposal. Failed or
361
463
partial attempts keep only measurements actually observed. After three
362
464
unsuccessful attempts there are no further operation markers for that key;
363
-
`firstSessionList` and `startupSettled` still report a later first success.
465
+
`firstSessionList`, `startupSettled`, and `firstSessionDiscoveryRegistration`
466
+
still report a later first success. `firstSessionDiscoveryResult` similarly
467
+
does not depend on a sampled scan handle.
364
468
A start without
365
469
an end can indicate in-flight work, a crash, a consent change, or delivery loss;
366
470
it is not a successful zero-duration operation. Hard process
0 commit comments