You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Cloud: keep content checksum and storage ETag in separate entity columns #1644
Someday/maybe. Found while testing Basic Memory Cloud readiness on development (2026-10-01). The user-visible symptom, empty search reported as "never indexed", is fixed separately by the readiness "ever indexed" rule.
Problem
On cloud, entity.checksum holds two different checksums, depending on which path wrote it last:
Path
Writes
accepted write (write_note / edit_note)
NULL (entity_repository.py create; the SHA-256 lives in note_content)
materialization
nothing (only mtime/size); the upload's ETag is discarded (put_file_if_unchanged returns a bool)
single-file / webhook index (index_current_markdown_file)
content SHA-256
full project index batch (S3FileInfo.checksum aliases etag)
S3 ETag
Both meanings are needed:
Content SHA-256: accepted-write CAS, live-update and provenance matching (storage_object_checksum_for_index_match compares bm-file-checksum with the indexed checksum), guarded deletes.
Storage ETag: cheap change detection from a listing (full index runs, project status).
Effects
Cloud project status (GET /v2/projects/{id}/status) compares listing ETags with entity.checksum, so notes indexed through the write or webhook path count as pending forever. Readiness on cloud never reaches idle.
Every webhook index and full pass re-downloads and re-hashes such notes, because the comparison never matches.
Proposed fix
Add nullable entity.storage_checksum, the checksum storage reports (S3 ETag on cloud, the SHA-256 locally), and keep entity.checksum as the content SHA-256 everywhere.
Writers:
materialization, from the upload response;
single-file indexing, from the GET that returned the bytes;
full index, from the listing.
Readers: full-index change detection and readiness compare listing against storage_checksum. NULL means unknown: fall back to today's behavior, and count as indexed in readiness.
Fills itself as notes are touched; no reindex needed. Optional cheap backfill:
copy rows where checksum already equals the listing ETag;
otherwise use one HEAD request for bm-file-checksum, and if it equals checksum, take the ETag.
A partial start, adding storage_checksum to RuntimeWrittenFileState and recording it on materialization, was drafted and set aside. It is not in any branch.
Type: Enhancement. It's a data-model cleanup for an inconsistency, and the user-visible symptom is already fixed separately.
Component: Cloud. It touches sync/indexing and the database schema, and needs an Alembic migration.
Priority: Low. The issue itself calls it someday/maybe. Cloud project status can count notes as pending and webhook or full-index passes re-download and re-hash notes. That is wasted work, not data loss.
Complexity: Medium to complex. It needs a new nullable entity.storage_checksum column and migration. The materialization, single-file index and full-index writers all have to populate it. Change detection and readiness have to read it, and NULL has to fall back to today's behavior. put_file_if_unchanged also has to return the ETag instead of a bool. The optional backfill adds more work.
Duplicates: A search of existing issues found none.
The issue's description of the problem and the proposed fix are clear. I have no clarifying questions. Per the triage scope, I haven't started any implementation.
Activity
claude commentedon Oct 2, 2026
Claude finished @phernandez's task in 10s —— View job
Triage
entity.storage_checksumcolumn and migration. The materialization, single-file index and full-index writers all have to populate it. Change detection and readiness have to read it, and NULL has to fall back to today's behavior.put_file_if_unchangedalso has to return the ETag instead of a bool. The optional backfill adds more work.The issue's description of the problem and the proposed fix are clear. I have no clarifying questions. Per the triage scope, I haven't started any implementation.