Today, 29 March 2017, at CloudNativeCon + KubeCon Europe in Berlin, the Technical Oversight Committee of the Cloud Native Computing Foundation (CNCF) accepted containerd as an incubating project. From here on, the low-level runtime that actually runs a container stops being an internal detail of Docker and becomes a component with a life of its own.

Context

Until 2016, saying “container” meant Docker: a single install holding together a command-line interface (CLI), a daemon, image building, registry handling, networking and the actual execution of isolated processes. When Kubernetes became the reference orchestrator, that bundling began to weigh. Kubernetes has no need for Docker’s CLI or docker build: it needs to start, stop and supervise containers, pull images and manage namespaces. Everything else, seen from the kubelet, is surface area nobody asked for.

Docker’s answer was to carve out its own core. In December 2016 it turned containerd — until then an internal component limited to execution — into a standalone project with a defined remit: image push and pull, low-level storage, container execution and supervision, network namespace management. Image building, logging volumes and the higher-level functions of the Docker daemon were left out by design. Today’s donation to the CNCF puts that perimeter in the hands of a neutral foundation.

Container Runtime Interface

What makes all of this possible is the Container Runtime Interface (CRI), which arrived as alpha in Kubernetes 1.5 on 19 December 2016. CRI is a gRPC and Protocol Buffers API with which the kubelet talks to the runtime, over a Unix socket where the kubelet is the client and the runtime (or a shim in its place) is the server.

The API defines two services:

  • RuntimeService, which manages the lifecycle of pods and containers and the interactive operations (exec, attach, port-forward);
  • ImageService, which pulls, inspects and removes images.

Before CRI, every runtime had to be wired into the kubelet by recompiling it. With CRI the kubelet speaks a stable protocol, and anything that implements it becomes a usable runtime. In Kubernetes 1.5 even Docker was moved behind this interface through a built-in shim, the dockershim, which translates CRI calls to the Docker daemon. Docker, in short, has already become one CRI runtime among others rather than the only one available.

Two implementations

Two answers are growing on top of the CRI layer, with different centres of gravity.

containerd exposes a gRPC API of its own and leans entirely on the Open Container Initiative (OCI) specifications and the runc reference implementation for the actual execution. Its ambition is to remain a general-purpose runtime, usable under Docker and under Kubernetes (with a CRI adaptation layer) and, in time, by other systems too. The published roadmap sets June 2017 for finishing the 1.0 work; the project states its intent to pursue OCI certification once it is available. Engineers from Alibaba, AWS, Google, IBM, Microsoft, Rancher, Red Hat and VMware work on the design effort.

CRI-O — which started as OCID, the Open Container Initiative Daemon, in September 2016 and was later renamed — sits in the Kubernetes incubator and is led by Red Hat. Its design choice is the opposite in breadth: implement CRI and nothing more. It is not a general runtime with a CRI adapter mounted on top; it is a daemon whose sole purpose is to run Kubernetes workloads. It too leans on runc (and in principle on any OCI-conformant runtime) for execution, pulls and unpacks OCI images, manages pod sandboxes, the container lifecycle, and a streaming server for kubectl exec, attach and logs. It is still pre-1.0.

The crux

The interesting difference is not which binary is smaller, but where each project draws its boundary of responsibility.

containerd bets that a reusable, general-purpose runtime exists, and that CRI is one of its consumers. That makes it useful outside Kubernetes too — it is already the engine inside Docker — at the price of maintaining its own API alongside the CRI adaptation. CRI-O bets the reverse: a container in a Kubernetes cluster needs no abstraction beyond CRI, and any extra surface is just more code to maintain. Its boundary coincides exactly with the API the kubelet already knows how to speak.

Both choices depend on the stability of the underlying OCI specifications, which as of today have not yet reached a final 1.0: the runtime spec and the image spec are at release-candidate stage. Until those two documents settle, any “OCI-conformant” runtime is chasing a target that is still moving.

Implications

For anyone running clusters today the practical consequence is limited. The normal path remains the kubelet with the dockershim in front of Docker, and images are in OCI format regardless of which runtime executes them. What changes is the direction: CRI has made the runtime a replaceable component, and the donation of containerd to a neutral foundation takes it out of the control of any single vendor. For an orchestrator that wants to stay vendor-independent, that is a precondition rather than a detail.

It is worth keeping two layers apart that often get conflated. CRI is the contract between kubelet and runtime. OCI is the contract on image format and on the behaviour of the executing runtime (runc and the like). containerd and CRI-O live in between: they speak CRI upward and OCI downward. It is this double hook that makes the pieces interchangeable.

Limits

The above is a snapshot of 29 March 2017. containerd has been incubating since today and its 1.0 is a roadmap, not a release. CRI-O is pre-1.0, and I have not yet seen production adoption figures to cite. The OCI specifications are at release candidate. Stronger claims — which runtime will become the default in which distribution, when the dockershim will be removed — would today be predictions rather than facts, and I make none. What is verifiable is the structural change: the runtime has been extracted, standardised behind CRI, and handed to shared governance.

Cover image: Three-column diagram comparing traditional deployment, virtual machines and containerization, showing the hardware, operating system… — diagram by MoreInput, CC BY-SA 4.0 — https://commons.wikimedia.org/wiki/File:Containerization2.svg