feat: Add circuit breaker for file sync failures by phernandez · Pull Request #364 · basicmachines-co/basic-memory · GitHub
Skip to content

feat: Add circuit breaker for file sync failures - #364

Merged
phernandez merged 1 commit into
mainfrom
feature/sync-circuit-breaker
Oct 16, 2025
Merged

feat: Add circuit breaker for file sync failures#364
phernandez merged 1 commit into
mainfrom
feature/sync-circuit-breaker

Conversation

@phernandez

Copy link
Copy Markdown
Member

Summary

Implements a circuit breaker pattern to prevent infinite retry loops when files consistently fail to sync. After 3 consecutive failures, files are automatically skipped until they're modified.

Changes

Core Implementation

  • Circuit breaker in sync_service.py: Tracks file failures with checksums (SHA-256)
    • MAX_CONSECUTIVE_FAILURES = 3 threshold
    • FileFailureInfo dataclass tracks failure count, timestamp, error, and checksum
    • Automatic reset when file content changes (new checksum)
    • Clears failure tracking on successful sync

Schema Updates

  • SkippedFileResponse: New Pydantic model for API responses
    • path: File path relative to project root
    • reason: Error message from last failure
    • failure_count: Number of consecutive failures
    • first_failed: Timestamp of first failure
  • SyncReportResponse: Added skipped_files field

CLI Enhancement

  • status command: Displays skipped files in both modes
    • Non-verbose: Summary count (e.g., "⚠️ 3 files skipped due to repeated failures")
    • Verbose: Detailed list with paths, failure counts, and error messages

Testing

  • 5 comprehensive circuit breaker tests:
    • test_circuit_breaker_skips_after_three_failures
    • test_circuit_breaker_resets_on_file_change
    • test_circuit_breaker_clears_on_success
    • test_circuit_breaker_tracks_multiple_files
    • test_circuit_breaker_handles_checksum_computation_failure
  • All use textwrap.dedent() for readable test content
  • Tests verify proper entity title extraction from frontmatter

Design Decisions

  1. Checksum-based reset: Files are retried when content changes, detected via SHA-256 checksum
  2. Simple threshold: 3 failures is a balance between resilience and preventing infinite loops
  3. Existing infrastructure: Leverages SyncReport.skipped_files instead of creating new status mechanisms
  4. User visibility: Surfaces failures in CLI status command where users check sync state

Testing

just test  # All 34 sync tests pass
just typecheck  # Type checking passes

Example Output

Non-verbose mode:

main: Status
├── docs/ +2 new ~1 modified
└── ⚠️  2 files skipped due to repeated failures

Verbose mode:

main: Status
└── ⚠️  Skipped (Circuit Breaker)
    ├── problematic/file1.md (failures: 3, reason: Invalid frontmatter)
    └── errors/file2.md (failures: 5, reason: Cannot read file)

Closes #189

🤖 Generated with Claude Code

Implements a circuit breaker pattern to prevent infinite retry loops
when files consistently fail to sync. After 3 consecutive failures,
files are skipped until they're modified.

Changes:
- Add circuit breaker in sync_service.py with MAX_CONSECUTIVE_FAILURES = 3
- Track file failures with checksums to detect when files change
- Add SkippedFile dataclass to report skipped files with failure details
- Update SyncReportResponse schema to include skipped_files field
- Display skipped files in CLI status command (summary + verbose modes)
- Add comprehensive tests for all circuit breaker scenarios

The circuit breaker:
- Tracks failures per file with SHA-256 checksums
- Resets automatically when file content changes
- Clears failure tracking on successful sync
- Prevents blocking sync operations on problematic files
- Logs warnings for skipped files

Fixes #189

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
@github-actions

github-actions Bot commented Oct 16, 2025

Copy link
Copy Markdown
Contributor

@phernandez
phernandez merged commit 434cdf2 into main Oct 16, 2025
16 checks passed
@phernandez
phernandez deleted the feature/sync-circuit-breaker branch October 16, 2025 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant