[release/2.2] core/unpack: fetch layers of every config-sharing manifest by chrishenzie · Pull Request #14140 · containerd/containerd · GitHub
Skip to content

[release/2.2] core/unpack: fetch layers of every config-sharing manifest - #14140

Merged
samuelkarp merged 1 commit into
containerd:release/2.2from
chrishenzie:unpack-shared-config-2.2
Sep 11, 2026
Merged

samuelkarp merged 1 commit into
containerd:release/2.2from
chrishenzie:unpack-shared-config-2.2

Conversation

@chrishenzie

@chrishenzie chrishenzie commented Sep 10, 2026 •

Copy link
Copy Markdown
Member

An index can carry more than one manifest that shares a config. A config lists uncompressed layer digests, so packaging the same layers with different compression (which podman manifest push --add-compression does) produces manifests with identical configs and the same config digest.

Right now, when the unpacker visits a manifest, it stores its layers in a map keyed by its config's digest, so they can be unpacked later once the config descriptor is visited. Because that map holds only one manifest's layers per digest, when more than one manifest shares a config, only the last visited manifest's layers are stored (and unpacked), and the others' layers are never fetched. Image pulls still report success, but a later export or push of the image fails with NotFound.

Fix this by changing the map to store the layers for all manifests pointing at a specific config (instead of just one manifest). This means manifests sharing the same config result in more than one set of layers being stored for the same config digest. Because manifests that share a config unpack to the same snapshot chain, we only need to unpack the first manifest's layers, and fetch the others (so their blobs reach the content store). Unpack skips fetching a layer whose snapshot already exists, so once a second manifest shares the config, the first manifest's layers are also fetched explicitly; a config named by a single manifest keeps that skip. Note that for indexes with more than one config-sharing manifest, this will now result in additional download time and storage for those layers.

Don't store anything for a manifest with no layers; an empty slice reaching the unpack would fail the pull.

(cherry picked from commit 3c5d9fe)

Include the registry test helper needed by the regression tests.

Assisted-by: Claude Code

Ensure all layers are fetched when multiple manifests in an index share a config descriptor

An index can carry more than one manifest that shares a config. A config
lists uncompressed layer digests, so packaging the same layers with
different compression (which `podman manifest push --add-compression`
does) produces manifests with identical configs and the same config
digest.

Right now, when the unpacker visits a manifest, it stores its layers in
a map keyed by its config's digest, so they can be unpacked later once
the config descriptor is visited. Because that map holds only one
manifest's layers per digest, when more than one manifest shares a
config, only the last visited manifest's layers are stored (and
unpacked), and the others' layers are never fetched. Image pulls still
report success, but a later export or push of the image fails with
NotFound.

Fix this by changing the map to store the layers for all manifests
pointing at a specific config (instead of just one manifest). This means
manifests sharing the same config result in more than one set of layers
being stored for the same config digest. Because manifests that share a
config unpack to the same snapshot chain, we only need to unpack the
first manifest's layers, and fetch the others (so their blobs reach the
content store). Unpack skips fetching a layer whose snapshot already
exists, so once a second manifest shares the config, the first
manifest's layers are also fetched explicitly; a config named by a
single manifest keeps that skip. Note that for indexes with more than
one config-sharing manifest, this will now result in additional download
time and storage for those layers.

Don't store anything for a manifest with no layers; an empty slice
reaching the unpack would fail the pull.

(cherry picked from commit 3c5d9fe)

Include the registry test helper needed by the regression tests.

Assisted-by: Claude Code
Signed-off-by: Chris Henzie <chrishenzie@gmail.com>
@github-project-automation github-project-automation Bot moved this from Needs Triage to Review In Progress in Pull Request Review Sep 11, 2026
@samuelkarp
samuelkarp merged commit 9f4eff5 into containerd:release/2.2 Sep 11, 2026
51 checks passed
@github-project-automation github-project-automation Bot moved this from Review In Progress to Done in Pull Request Review Sep 11, 2026
@chrishenzie
chrishenzie deleted the unpack-shared-config-2.2 branch September 12, 2026 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants