{{ message }}
Repository navigation
Conversation
A non-root page split (rumPlaceToPage, and rumInsertValueOld when documentdb_rum.track_incomplete_split is off) registered the page it got from RumNewBuffer() for generic WAL with flags 0, so it was logged as a delta against its previous contents. When the page was just extended it is all-zero on disk until written out. With full_page_writes = off, crash recovery's XLogReadBufferForRedo sees a new page, generic_redo skips the delta, and crash recovery doesn't run the invalid-page check, so it ends without an error and leaves downlinks and sibling links pointing at all-zero pages. Later writes that descend into one of them never return (documentdb#807). Register the new right page with GENERIC_XLOG_FULL_IMAGE, as the root split, createPostingTree, RumInitBuffer and RumInitMetabuffer already do for their new pages. The left page and the right-right sibling already exist and stay deltas. With full_page_writes = on nothing changes: a never-written page already got a full-page image. RumNewBuffer()'s comment now states that callers must log the page as a full image. Adds 004_rum_split_full_page_writes_off to the RUM recovery TAP tests. It loads two collections, one per split code path, with full_page_writes = off, no bgwriter writes and no checkpoints, crashes the server and recovers it, then checks that no index block is all-zero, that an insert completes, and that index scans match sequential scans. Without the fix both split paths leave zero pages behind. Fixes documentdb#807 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018uo1BA8DhzNJHxezANds93 Signed-off-by: Hari Krishna Sunder <hari90@users.noreply.github.com>
A RUM index damaged as in documentdb#807 (or any other way) has links to all-zero pages. A zero page has no flags, so the descent in rumFindLeafPage() takes it for an internal page, findChildPage() returns block 0, the metapage, which is also not a leaf, and the loop never ends. It never checks for interrupts either, so the backend spins at 100% CPU and ignores pg_cancel_backend and pg_terminate_backend. (An assert-enabled build trips an assertion in PageGetSpecialPointer instead.) Add rumCheckTreePage(), which raises ERRCODE_INDEX_CORRUPTED with a REINDEX hint for an all-zero page or the metapage, and call it under the lock, before the page's flags are read, wherever a page is reached through a tree link: rumTraverseLock() (each level of the descent), rumStepCore() and rumStepEntryForScans() (sibling steps), the rumFindParents() walk, the "refind the leftmost page" loops in entryGetNextItem() and entryFindItem(), and VACUUM's rumFindLeftMostLeafPage(). rumFindLeafPage() also raises it for a page of the other tree kind (entry vs. posting) and for an invalid or self-referencing downlink, which were only Asserts, and calls CHECK_FOR_INTERRUPTS() on each level. The sibling type-mismatch and deleted-sibling errors use ERRCODE_INDEX_CORRUPTED too. "Lost path" stays an internal error: GetWriteErrorFromErrorData() turns it into the retryable ERRCODE_INDEX_LOSTPATH. Extends 004_rum_split_full_page_writes_off: it zeroes every posting-tree leaf page of one index on disk and checks that an insert and an index scan fail promptly with "contains unexpected zero page". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018uo1BA8DhzNJHxezANds93 Signed-off-by: Hari Krishna Sunder <hari90@users.noreply.github.com>
visridha
reviewed
Oct 21, 2026
Contributor
There was a problem hiding this comment.
The rest of the change looks good to me - I'm just curious if this is actually safe - not sure i follow the comment of "where the lock holds off interrupts" - coudl you clarify this please?
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
Fixes #807. When a RUM page that isn't the root splits, the new right page is WAL-logged as a generic delta instead of a full image. With
full_page_writes = off, crash recovery then loses the page. The server reports nothing, and the index is left with downlinks and sibling links to all-zero pages. Any later write that descends into one of those pages spins forever and can't be cancelled.Commit 1: log the new right page as a full image
Root cause:
rumPlaceToPage()(the default path) andrumInsertValueOld()(used whendocumentdb_rum.track_incomplete_split = off) registered the page fromRumNewBuffer()withGenericXLogRegisterBuffer(..., 0). A page that was just extended is all-zero on disk. During crash recovery,XLogReadBufferForRedosees a new page,generic_redoskips the delta, and the invalid-page check doesn't run.GENERIC_XLOG_FULL_IMAGE. The root split,createPostingTree,RumInitBufferandRumInitMetabufferalready do this for their new pages. I checked everyRumNewBuffer()caller; these two were the only ones logging a delta.XLOG_GENERICrecord. WAL volume is about the same. Withfull_page_writes = ona never-written page already got a full-page image, and with it off a delta against zeros is already close to page size.RumNewBuffer()'s comment now states that its pages must be logged as full images.REINDEX.Commit 2: raise index corruption instead of hanging
A zero page has no flags, so the descent treats it as an internal page.
findChildPage()then returns block 0, the metapage, which is also non-leaf, so the loop never ends. It never checked for interrupts.rumCheckTreePage()raisesERRCODE_INDEX_CORRUPTEDwith a REINDEX hint for an all-zero page or the metapage. It runs under the lock, before the page's flags are read, wherever a page is reached through a tree link:rumTraverseLock(), for every level of the descentrumStepCore()andrumStepEntryForScans(), for sibling stepsrumFindParents()walkentryGetNextItem()andentryFindItem()rumFindLeftMostLeafPage()PageGetSpecialPointer's assertion.rumFindLeafPage()also errors on a page of the other tree kind (entry vs. posting) and on an invalid or self-referencing downlink, which were only Asserts. It also callsCHECK_FOR_INTERRUPTS()on each level.ERRCODE_INDEX_CORRUPTED."Lost path"deliberately stays an internal error.GetWriteErrorFromErrorData()matches it and turns it into the retryableERRCODE_INDEX_LOSTPATH.Tests
New TAP test
pg_documentdb/src/test/extended_rum_recovery_tests/t/004_rum_split_full_page_writes_off.pl:full_page_writes = off, turns off bgwriter writes, checkpoints and autovacuum, and runs one collection per split path (track_incomplete_spliton and off).stop('immediate')and lets it recover.Verified in the dev-container image (PG 16, assert-enabled):
make -C pg_documentdb check-extended-rum-recovery(001–004)check-extended-rumcheck-extended-rum-builtin-rmgrcheck-regresscitus_indentis clean.Type of Change
AI Disclosure
Checklist
git commit -s)🤖 Generated with Claude Code
https://claude.ai/code/session_018uo1BA8DhzNJHxezANds93
cloud.mekodata.ai://48fd42c0-7b11-4c27-a0a9-db6f5be829d7;adlc/proj/documentdb/documentdb
meko_design:design-fix-807.md;62a752e216b8f7a654d2997acffb49da93abf5067a7b5039adc331c7fd376bb7
meko_conversation_id:54978901058b4bafa64ae3b040649dff