HBASE-30480 Backup code mixes up region server casing by DieterDP-ng · Pull Request #8784 · apache/hbase · GitHub
Skip to content

HBASE-30480 Backup code mixes up region server casing - #8784

Open
DieterDP-ng wants to merge 1 commit into
apache:masterfrom
DieterDP-ng:HBASE-30480_backup-region-server-casing
Open

DieterDP-ng wants to merge 1 commit into
apache:masterfrom
DieterDP-ng:HBASE-30480_backup-region-server-casing

Conversation

@DieterDP-ng

Copy link
Copy Markdown
Contributor

The backup code keys per-server state by host:port. WAL file names contain the lowercased hostname, while ServerName#getAddress keeps the hostname's original case. For a region server whose hostname contains a capital letter, the backup code therefore tracked the server under two keys, and lookups between them missed. Incremental backups then fell back to the oldest boundary of any server, so they could include WALs that an earlier backup already covered. Replaying those WALs could bring back deleted data, or fail the restore when the WALs held edits for a column family that was dropped since. Boundaries were also stored under the wrong key for later backups. The log cleaner missed the host's boundary and kept WALs longer than needed. Various tests were broken when running on a host with a capital letter.

BackupUtils#toHostKey now lowercases the key. The WAL path parsers, the log roll (both variants) and the live-server check of the full backup use it. Reads from the backup system table and BackupBoundaries also normalize, because existing tables can hold mixed-case keys. On a collision, the newest roll result and the oldest boundary are kept.

Mixed-case keys that are already stored are phased out as follows:

  • The per-table boundaries (trslm rows) are rewritten with lowercase keys by the next backup that includes the table. A table that is not backed up again keeps its old row. Backups with continuous backup enabled do not rewrite this row.
  • The roll results (rslogts rows) are never phased out, because nothing deletes these rows. Once the host rolls its WALs on the new code, its lowercase row holds the larger value, and the read keeps that value.
  • The boundaries copied into each backup's BackupInfo keep their mixed-case keys until that backup is deleted. BackupBoundaries normalizes them.

The legacy log roll procedure looked up the previous roll result by hostname without port, so that lookup never matched. It now uses the full key.

TestBackupOfflineRS now runs with a mixed-case region server hostname, so it covers this on hosts with lowercase names too.

The backup code keys per-server state by host:port. WAL file names
contain the lowercased hostname, while ServerName#getAddress keeps the
hostname's original case. For a region server whose hostname contains a
capital letter, the backup code therefore tracked the server under two
keys, and lookups between them missed. Incremental backups then fell
back to the oldest boundary of any server, so they could include WALs
that an earlier backup already covered. Replaying those WALs could
bring back deleted data, or fail the restore when the WALs held edits
for a column family that was dropped since. Boundaries were also stored
under the wrong key for later backups. The log cleaner missed the
host's boundary and kept WALs longer than needed. Various tests were
broken when running on a host with a capital letter.

BackupUtils#toHostKey now lowercases the key. The WAL path parsers, the
log roll (both variants) and the live-server check of the full backup
use it. Reads from the backup system table and BackupBoundaries also
normalize, because existing tables can hold mixed-case keys. On a
collision, the newest roll result and the oldest boundary are kept.

Mixed-case keys that are already stored are phased out as follows:
- The per-table boundaries (trslm rows) are rewritten with lowercase
  keys by the next backup that includes the table. A table that is not
  backed up again keeps its old row. Backups with continuous backup
  enabled do not rewrite this row.
- The roll results (rslogts rows) are never phased out, because nothing
  deletes these rows. Once the host rolls its WALs on the new code, its
  lowercase row holds the larger value, and the read keeps that value.
- The boundaries copied into each backup's BackupInfo keep their
  mixed-case keys until that backup is deleted. BackupBoundaries
  normalizes them.

The legacy log roll procedure looked up the previous roll result by
hostname without port, so that lookup never matched. It now uses the
full key.

TestBackupOfflineRS now runs with a mixed-case region server hostname,
so it covers this on hosts with lowercase names too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DieterDePaepe

Copy link
Copy Markdown

@DieterDePaepe
DieterDePaepe requested a review from hgromer October 16, 2026 19:20
@DieterDePaepe

Copy link
Copy Markdown

Test failure is related to TestPointInTimeRestore (continuous backups), not this PR.

@DieterDePaepe

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants