fix(build, stages): avoid full tag scans for digest lookups by alexey-igrychev · Pull Request #8075 · werf/werf · GitHub
Skip to content

fix(build, stages): avoid full tag scans for digest lookups - #8075

Draft
alexey-igrychev wants to merge 1 commit into
mainfrom
fix/build/digest-tag-lookup-v3
Draft

fix(build, stages): avoid full tag scans for digest lookups#8075
alexey-igrychev wants to merge 1 commit into
mainfrom
fix/build/digest-tag-lookup-v3

Conversation

@alexey-igrychev

@alexey-igrychev alexey-igrychev commented Oct 15, 2026

Copy link
Copy Markdown
Member

Summary

Fresh stage lookups download the entire repository tag inventory even when they need one build digest. Request a bounded ordered range for that digest on eligible registries while retaining authoritative checks and compatibility fallback.

What

  • Send authenticated tags/list requests with n=100 and a digest cursor for fresh lookups on eligible registries; retain bare digest tags, exact digest matching, rejected markers and parent creation timestamps.
  • Fall back to full listing for unsupported providers or incompatible range responses. Authentication, throttling and server failures remain errors.
  • Make observed stages and rejected markers visible to later cached consumers and callers awaiting an older full listing, without treating a partial range as complete inventory or refreshing the full snapshot's age.
  • Unrelated digests published externally after the initial snapshot can still cause a duplicate build before their own authoritative lookup; full fallback responses do not refresh the wrapper's entire cached inventory.
  • Publication locks, fresh-request boundaries, stage digests/timestamps, CLI flags and import-server behavior remain unchanged.
  • UNVERIFIED: full-build wall-clock improvement after accounting for cache freshness, duplicate work and registry implementation costs; a representative before/after build comparison is required before accepting this optimization.

Why

Publication and recovery decisions need a response newer than their decision boundary. Reusing an old listing can hide a concurrent winner, so TTL or joining an older request cannot replace that check. Fetching unrelated tags is not required to select a stage for one digest, but narrowing the response changes incidental whole-cache refresh behavior and requires explicit observation merging and conservative registry compatibility handling.

@alexey-igrychev
fix(build, stages): avoid full tag scans for digest lookups
562657d 
Keep publication and recovery lookups authoritative while reading only the relevant ordered tag range. Preserve incompatible-registry fallback and merge observed winners and rejections into cached inventory.

Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
@alexey-igrychev

alexey-igrychev commented Oct 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

@alexey-igrychev alexey-igrychev mentioned this pull request Oct 15, 2026
Closed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alexey-igrychev