Summary

Replace Moby image backend with containerd. This will move ownership of image related code outside of Moby allowing for a large code and scope reduction in Moby. Many refactorings can be done before doing the switch, but changing the backend can only be done in large increments due to the metadata storage architecture difference in containerd.

Overview

• Image storage moved to containerd metadata database
• Migration of existing metadata
• All image requests use containerd client
• Distribution code removed and replaced with containerd client
• Containerd’s snapshotters supported
• Graph drivers scoped down (no more mounting/diffing)
• Graph drivers supported as containerd snapshotter plugins

Plan

This change is deep, in some cases we can completely bypass existing packages and use containerd (i.e. distribution), while in others we will need to write new code to integrate with existing Docker interfaces. There will likely be a few large PRs since Docker will not be able to be in a functional state unless the data storage is consistently used by all commands. This plan is an attempt to break down the work in individual PRs which can be merged in isolation while preserving as much functionality as possible. Some PRs may temporarily break upgrades since we will want to begin testing and integrating with some of these changes before the upgrade migration code is complete.

I have broken this down into phases to better understand the relationship between work items. There is no definition for the end of a phase but rather the work in each phase should be able to be completed concurrently.

Phase 1

• libcontainerd simplification and windows runtime integration
• daemon updates
  • replace plugin executor interface with containerd client
  • replace image backend and distribution with containerd client
    • includes all image operations: ls, rm, pull, push, etc
    • unpack from content store into layer store
• minimizing graph driver scope
  • move mount to layer store and utilize containerd mount package
  • move diff to layer store and utilize containerd diff packages

Phase 2

• move libcontainerd into daemon
• image metadata migration
  • support migrating existing docker metadata into containerd metadata store
• graph drivers updated to snapshotter interface
  • layer store communicating with containerd snapshotter

Phase 3

• replace layer store with snapshotter
  • graph drivers exposed as containerd snapshotter plugins
  • support layer recreation into content store from tar split on push
• update storage driver configuration to reflect snapshotter preference

Related PRs

• [wip] Update containerd runtime to v1.2.0 #37932 Update containerd to 1.2x #38168 Update containerd to 1.2x
• Remove the rest of v1 manifest support #37874 Remove the rest of v1 manifest support
  • Keep but deprecate registry v2 schema1 logic and revert to libtrust-key-based engine ID #39365 Keep but deprecate registry v2 schema1 logic and revert to libtrust-key-based engine ID
  • Remove v2 schema1 push #39384 Remove v2 schema1 push
  • Remove more registry v1 code #39371 Remove more registry v1 code
• Containerd image and snapshot backend integration #38738 Containerd image and snapshot backend integration

Related project

https://github.com/dmcgowan/docker/projects/1 containerd-integration