Dockershim removal feedback & issues

[mrbobbytables]

Please use this issue for feedback and issues regarding dockershim removal in the 1.24 release.

/triage accepted
/milestone v1.24
/sig docs node
/area docker

[afbjorklund]

There is still no release available for cri-dockerd, even though being announced in April.

https://www.mirantis.com/blog/the-future-of-dockershim-is-cri-dockerd/

But except for having to (patch and) build it yourself, it works OK with the Docker runtime.

$ cat /etc/crictl.yaml
runtime-endpoint: unix:///var/run/cri-docker.sock
image-endpoint: unix:///var/run/cri-docker.sock

UPDATE: The path has now been changed, to cri-dockerd.sock

Mirantis/cri-dockerd@50c048c

---

Currently the CRI instructions are missing from the documentation:

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#docker

So the instructions only work with 1.23 and earlier releases, not with 1.24.

sudo systemctl daemon-reload
sudo systemctl enable cri-docker.socket
sudo systemctl start cri-docker.socket

The added requirements of installing crictl and configuring socket location is also "harder" than before.
This is due to kubeadm only detecting dockershim and containerd and cri-o automatically, not cri-dockerd.

[afbjorklund]

Please use this issue for feedback and issues regarding dockershim removal in the 1.24 release.

This information is confusing: https://kubernetes.io/docs/tasks/administer-cluster/migrating-from-dockershim/

"This section presents information you need to know when migrating from dockershim to other container runtimes."

Shouldn't it be "when migrating from dockershim to other CRI implementations for Docker" ?

i.e. going from dockershim to cri-dockerd

"It is recommended to migrate from dockershim to alternative container runtimes."

Shouldn't it be "it is recommended to migrate from dockerd to alternative container runtimes" ?

i.e. going from dockerd to containerd

---

We saw the same thing happen with CRI-O, before. As the saying goes: "now you have two problems".

It is especially confusing when the docs still installs containerd from docker.com instead of containerd.io.

Moving from dockershim to cri-dockerd should be mandatory, since CRI will be required in 1.24 and up.

Moving from dockerd to containerd (or cri-o) could still be optional, and requires more user intervention.

[neolit123]

A transition to cri-dockerd first might be smoother, indeed. There is especially a concern for business with a lot of clusters, with a lot of docker nodes. If they cannot move to alternative runtimes due to bugs, these bugs will have to be reported and resolved. Requiring things like a host OS update with the CR update is not great.

cri-dockerd is currently a third party project with an undefined maintenance state. Because of that it is difficult to maintain a k8s.io guide for it. The new guide for migrating to containerd seems like the best option for the time being.

At the same time we don't want users to be stuck with a 1.23 kubelet. Note: the k8s skew policy allows nodes to continue to have a 1.23 kubelet against a 1.24 and 1.25 kube-apiserver, but that would only delay the required changes.

https://kubernetes.io/releases/version-skew-policy/#kubelet

[afbjorklund]

If I understand correctly, the plan is to remove all support (for docker) from the documentation and from the supporting tools like kubeadm and leave users to make their own adventure (or to pay for Docker or Mirantis)

• No documentation on kubernetes.io
• No testing in kubernetes regression
• No binary packages available
• No support in kubeadm init

For minikube it seems like continuing with docker runtime is doable, even if it means having to build our own CRI binaries and do our own CRI configuration. We do have containerd available as opt-in, since a long time back.

• Use cri-dockerd instead of dockershim where needed minikube#12102
• Add cri-dockerd (and crictl) to installation minikube#12103
• consider using minikube image build GoogleContainerTools/skaffold#5739
• use minikube image load for containerd GoogleContainerTools/skaffold#5773

[neolit123]

If I understand correctly, the plan is to remove all support (for docker) from the documentation and from the supporting tools like kubeadm and leave users to make their own adventure (or to pay for Docker or Mirantis)

if cri-dockerd becomes a maintained and documented project, i don't see a problem for k8s.io to link and mention it as part of CR setup. https://kubernetes.io/docs/setup/production-environment/container-runtimes/

No documentation on kubernetes.io

cross-linking from k8s.io to cri-dockerd guides still make sense.

No testing in kubernetes regression

testing is difficult if the CRI implementation is external and unmaintained. if cri-dockerd ramp up on the maintenance i can see kubeadm adding back e2e tests for docker. we switched all test jobs to containerd for the time being.

No binary packages available

cri-dockerd need to start making releases and have a support matrix against the kubelet especially WRT the CRI API version that is in use by both.

No support in kubeadm init

kubeadm init|join allows you to pass any custom CRI socket via Init|JoinConfiguration.nodeRegistration.criSocket.
the CR will be treated as "remote" and configured (via a kubelet flags) similarly to how containerd | cri-o users, use kubeadm today. if docker.sock is present on the host and if cri-dockerd setups the legacy dockershim.sock, kubeadm would autodetect docker as the CR and start using it (we still have to make one minor change for kubeadm 1.24).

For minikube it seems like continuing with docker runtime is doable, even if it means having to build our own CRI binaries and do our own CRI configuration. We do have containerd available as opt-in, since a long time back

i would not recommend for minikube and users of k8s to start using cri-dockerd until the project is actually maintained, has docs and releases. if Mirantis cannot staff the project, they'd have to grant community ownership.

[afbjorklund]

I think Mirantis had plenty of time for that, with the delay... Don't think moving to 1.25 would have helped ?

Was referring to the auto-detection and the recommendations, it does work to use kubeadm with params

if cri-dockerd setups the legacy dockershim.sock

cri-dockerd now sets up /var/run/cri-docker.sock

EDIT: path changed to /var/run/cri-dockerd.sock

---

Users of minikube are free to choose their kubernetes version and the container runtime, using parameters..

If they want to stick with k8s 1.23, or move to containerd, they are free to do so. Otherwise, it is inevitable.
i.e. moving to version 1.24 and sticking with docker, without using cri-dockerd, is not going to be doable.

But it doesn't look to be too much worse than cri-o... Tested in another product only, scarce releases, etc.

[rphillips]

Linking my issue about local-up-cluster.sh not working nicely due to the dockershim removal: #106972