Repository navigation
test(runtime): add persistence observer streaming benchmarks - #4077
salignatmoandal wants to merge 4 commits into
Conversation
36cfbd2 to
084471c
Compare
|
👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push. Commits that are not verified: See GitHub's guide on signing commits for setup instructions. I've added |
5cbf63a to
e40f6b6
Compare
|
Hi @aheritier, thanks for the heads-up. I've signed the commit with my SSH signing key and force-pushed (e40f6b6). GitHub now shows it as verified on my side. Happy to adjust anything else if needed. |
Sayt-0
left a comment
There was a problem hiding this comment.
Useful addition: the streaming persistence contract (1 AddMessage + N-1 UpdateMessage per turn) was not pinned by any focused test, and both characterization tests verify correctly against persistStreamingContent. Tests pass locally and follow the surrounding conventions (t.Parallel, t.Context, testify, :memory: + SetMaxOpenConns(1) pattern from pkg/session/store_memory_test.go).
Main issue: the in-memory benchmark result is a function of b.N (details inline), which defeats the stated goal of establishing a baseline. The SQLite benchmark is fine (reproduced ~2.9 ms/op, stable across benchtime values).
| Area | Status |
|---|---|
| Characterization tests | correct, verified against persistence_observer.go |
| SQLite benchmark | stable, no change needed |
| In-memory benchmark | numbers depend on b.N, needs fix (inline comment) |
| Conventions, vet | clean |
Non-blocking: the file name persistence_observer_bench_test.go also hosts two unit tests; moving them to persistence_observer_streaming_test.go would keep the bench file benchmark-only, matching pkg/tui/components/message/bench_test.go.
|
Good catch — confirmed in Fixed in a0ddcd4 with your suggested harness: each iteration does Re-measured on darwin/arm64 (Apple M3 Pro):
Happy to follow up on the slog |
aheritier
left a comment
There was a problem hiding this comment.
The benchmark behavior looks sound, but this branch is 592 commits behind main and does not compile against it due to the helper collision and stale UpdateMessage signature above. There is also a lint failure. Please rebase onto current main, address these items, and rerun CI.
29b4b7f to
f66e8b1
Compare
Document the per-chunk AddMessage/UpdateMessage contract and establish in-memory vs SQLite baselines for streaming assistant persistence.
Reset the session each iteration so UpdateMessage's O(sessions×messages) scan cannot make ns/op a function of b.N.
Discard the default slog handler during SQLite store setup so harness re-invocations do not interleave migration INFO with -bench output.
Rename countingStore to avoid colliding with generated_file_test.go, forward sessionID through UpdateMessage, and satisfy embeddedstructfieldcheck.
f66e8b1 to
cfe2f18
Compare
aheritier
left a comment
There was a problem hiding this comment.
The bounded-store fix addresses the benchmark harness issue. The ~2.0 ms/op figure and related caveats still in the PR description describe the old unbounded harness, while your follow-up reports ~160 µs/op after the fix. Please rerun both benchmarks at the current head and refresh the sample results, per-chunk figures, store ratio, and resolved caveats. Include the SHA, Go version, hardware, and benchmark command so the comparison is reproducible. I have not independently reproduced these numbers; the improvement should be described as a harness correction, not a production-code speedup.
|
|
||
| finalizeStreamingMessage(ctx, obs, sess) | ||
| // MessageAdded with an existing streaming row issues one more UpdateMessage. | ||
| assert.Equal(t, int64(chunks), store.updateCalls.Load()) |
There was a problem hiding this comment.
Nonblocking: this checks the final UpdateMessage count, but not the canonical persisted payload or the next-turn reset. Please reload after MessageAdded and assert exactly one assistant row containing the final "done" payload, with no additional AddMessage. Then emit a second turn using the same observer/session and verify a new row and one additional AddMessage, preserving the first row and excluding prior-turn content/reasoning.
| require.NoError(t, err) | ||
| require.Len(t, reloaded.Messages, 2) // user + streaming assistant | ||
| require.NotNil(t, reloaded.Messages[1].Message) | ||
| assert.Equal(t, "hello", reloaded.Messages[1].Message.Message.Content) |
There was a problem hiding this comment.
Nonblocking: both new tests emit only AgentChoice text deltas, although the PR includes reasoning deltas in its characterization scope. Please add a reasoning-first sequence followed by interleaved reasoning/text deltas, asserting one AddMessage on the first delta, an UpdateMessage for each subsequent delta, exactly one assistant row, and independently accumulated Content and ReasoningContent after reload.
There was a problem hiding this comment.
Optional: this comment describes the old Store implementation. UpdateMessage now loads the specified session and scans its messages, rather than scanning every session. Please explain that reusing a session grows its message list, while creating/deleting a session per iteration bounds the per-session scan and retained store state.

Summary
Adds characterization tests and benchmarks for
PersistenceObserverstreaming persistence — the path that mirrors assistant token deltas (AgentChoice/AgentChoiceReasoning) into a single growing message row in the session store.This establishes a baseline before any future optimization (e.g. debounced flushes) and documents the per-chunk store write contract that the observer currently implements.
Context
During a streaming assistant turn, the runtime emits one
AgentChoiceEventper delta.PersistenceObserver.persistStreamingContentkeeps a single in-flight row and:AddMessage) on the first chunkUpdateMessage) on every subsequent chunkMessageAddedEventto finalise the row with the canonical payloadThis behaviour is easy to regress when refactoring persistence or the store layer, but was not previously covered by a focused unit test or benchmark.
Changes
New file:
pkg/runtime/persistence_observer_bench_test.goTests
TestPersistenceObserver_UpdateCountPerChunkAddMessage+(N-1)UpdateMessagecalls for N streaming chunks, plus one finalUpdateMessageonMessageAddedTestPersistenceObserver_StreamingContentAccumulates"hel"+"lo"→"hello")Both tests use a
countingStorewrapper aroundInMemorySessionStoreto assert store call counts without mocking the observer.Benchmarks
Each iteration simulates a long assistant turn: 500
AgentChoicedeltas + 1MessageAddedfinalisation (streamingBenchChunks = 500).BenchmarkPersistenceObserver_StreamingChunksInMemorySessionStoreBenchmarkPersistenceObserver_StreamingChunks_SQLite:memory:json.Marshal+UPDATE session_itemsper chunkSample results (darwin/arm64, Apple M3 Pro)
Per chunk (500 chunks/iter): ~4 µs in-memory, ~6 µs SQLite, ~5 vs ~17 allocs.
SQLite is ~1.5× slower and ~3× more alloc-heavy — expected given JSON marshal + SQL per update.
Known benchmark caveats
b.Niterations, soUpdateMessageeventually scans an ever-growing message list (O(messages) per update). Later iterations are slower than the first. SQLite stays flat because updates are keyed bymessage_id.NewSQLiteSessionStoreFromDB, which logs migration info beforeb.ResetTimer()— noisy stdout but not included inns/op.These caveats are acceptable for a baseline but worth keeping in mind when comparing future numbers.
Why now?
Streaming persistence is on the hot path for every assistant response when a session store is configured. Having explicit tests + benchmarks makes it safer to:
persistStreamingContent(avoidstrings.Builder.String()copies on every delta)Test plan
go test ./pkg/runtime -run TestPersistenceObserver_UpdateCountPerChunk -vgo test ./pkg/runtime -run TestPersistenceObserver_StreamingContentAccumulates -vgo test ./pkg/runtime -run=^$ -bench=BenchmarkPersistenceObserver -benchmem -count=1task test(full suite)task lint