docs(storage): host envelopes and post-1562 SurrealKV evidence by joshuajbouw · Pull Request #1571 · astrid-runtime/astrid · GitHub
Skip to content

docs(storage): host envelopes and post-1562 SurrealKV evidence - #1571

Open
joshuajbouw wants to merge 1 commit into
codex/1562-volume-wal-cachefrom
codex/1562-storage-bench-evidence
Open

docs(storage): host envelopes and post-1562 SurrealKV evidence#1571
joshuajbouw wants to merge 1 commit into
codex/1562-volume-wal-cachefrom
codex/1562-storage-bench-evidence

Conversation

@joshuajbouw

Copy link
Copy Markdown
Member

Linked Issue

Refs #1561
Refs #1398

Evidence for the existing WAL/cache rewrite and storage performance record. Not a new product issue.

Summary

Make storage I/O results reproducible for other people on two published profiles, and re-run the WAL/cache vs SurrealKV microbenchmark after #1562. This is an evidence envelope, not a CI gate, and not a merge of #1562.

Changes

  • Record provenance.host.{machine_class,filesystem,volume_kind,uname} on astrid-storage-io-benchmark-v2.
  • Pin the manually dispatched Storage benchmark evidence workflow to Rust 1.95.0 and the linux-ext4-gh profile (GitHub VM host path, not Docker overlay).
  • Add ignored-test sidecar format astrid-storage-kv-microbench-v1 when ASTRID_STORAGE_PERF_OUTPUT is set.
  • Document how to run, what the numbers mean, and what they do not mean. Never ASTRID_HOME=$HOME/.astrid.
  • Record macos-apfs-local transcript at 6066bca1.

Verification

  • cargo clippy -p astrid-storage --features legacy-surrealkv --lib --benches --tests -- -D warnings
  • cargo test -p astrid-storage --test storage_io_report — 4 passed
  • Ignored release-mode KV tests on clean 6066bca1 / Apple M2 Ultra APFS host path:
    • hot point reads median ratio 1.754 vs SurrealKV (faster)
    • concurrent 8-reader median ratio 12.459
    • working-set 4096 median ratio 1.616
    • WAL-on batch comparison did not finish after 88 CPU-minutes still in native apply_batch / load_kv_object_for
  • Transcript: docs/benchmarks/storage-io/wal-cache-vs-surrealkv-6066bca-macos-apfs.txt
  • Native microbenchmarks are not end-to-end and are not a CI gate.

AI / Tool Assistance

Assisted-by: OpenAI Codex: Grok 4.6
Assisted-by: OpenAI Codex: GPT-5.6-luna

Codex implemented harness/docs and ran the comparison. The human contributor remains responsible for the evidence boundaries and review responses.

Checklist

  • Linked to existing issues (Refs, not a new chore issue)
  • CHANGELOG.md updated (entry under [Unreleased])
  • I understand every change in this PR and can explain its design, risks, and validation.
  • I reviewed and tested any meaningful tool-generated output included in this PR.
  • Every non-bot, non-merge commit has a matching Signed-off-by trailer.

Extend storage_io v2 provenance with the measured host filesystem and
volume kind so linux-ext4-gh and macos-apfs-local runs can be reproduced
by other people. Keep the ignored WAL/cache vs SurrealKV tests as
microbenchmark evidence, not a CI gate.

WAL-off hot point reads at 6066bca on macos-apfs-local were faster than
SurrealKV (median 1.754x). Do not copy the old 0.947x figure. The WAL-on
batch comparison did not finish after 88 CPU-minutes in native
apply_batch.

Refs #1561
Refs #1398

Assisted-by: OpenAI Codex: Grok 4.6
Assisted-by: OpenAI Codex: GPT-5.6-luna

Signed-off-by: Joshua J. Bouw <jjb@unicity-labs.com>
Copilot AI balanced review requested due to automatic review settings August 20, 2026 11:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants