feat: implement core sequencer with gap-free sequence guarantee by vsai12 · Pull Request #17899 · bytebase/bytebase · GitHub
Skip to content

feat: implement core sequencer with gap-free sequence guarantee - #17899

Closed
vsai12 wants to merge 9 commits into
mainfrom
audit-sequencer
Closed

vsai12 wants to merge 9 commits into
mainfrom
audit-sequencer

Conversation

@vsai12

@vsai12 vsai12 commented Oct 29, 2025 •

Copy link
Copy Markdown
Contributor

Summary

Implements core audit log sequencer (PR # 2) with guaranteed gap-free sequence numbers (1, 2, 3...) for reliable log aggregation and integrity monitoring.

Key Features

  • Gap-free sequencing: Single-writer pattern ensures no gaps under concurrent load (verified with 10k events from 100 concurrent clients)
  • Backpressure handling: Non-blocking queue submission with error propagation when overloaded
  • Service layer: Clean separation of domain logic, data layer, and orchestration
  • UTF-8 safe validation: Payload size limits with character boundary preservation
  • Graceful shutdown: Context-based timeout coordination matching server patterns

Implementation Details

  • Config management with production defaults and comprehensive validation
  • Store method EnsureUniqueBytebaseID with collision retry (up to 5 attempts)
  • Database migration: UNIQUE constraint on (bytebase_id, sequence_number)
  • Error handling: common.Error with proper codes (Invalid, Internal, SizeExceeded)
  • Test coverage: 12 tests including race detector and concurrent load verification

Files Changed

  • backend/common/audit/: config.go, logger.go, service.go, validation.go, instance.go (refactored)
  • backend/store/audit_log.go: Added EnsureUniqueBytebaseID method
  • backend/migrator/migration/3.9/0011##audit_log_unique_bytebase_id.sql: UNIQUE index
  • Tests: 575 LOC comprehensive test suite

Dependencies

Requires PR #17897 (proto fields + Bytebase ID generation)

@cla-bot cla-bot Bot added the cla-signed label Oct 29, 2025
@github-actions

github-actions Bot commented Oct 29, 2025 •

Copy link
Copy Markdown

@d-bytebase
d-bytebase requested a review from h3n4l October 29, 2025 10:19
Comment thread backend/tests/audit_foundation_test.go Outdated
@vsai12
vsai12 marked this pull request as draft October 30, 2025 02:28
Comment on lines +30 to +32

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(PR #3)?

@vsai12 vsai12 closed this Nov 10, 2025
@vsai12
vsai12 deleted the audit-sequencer branch November 10, 2025 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants