Talk on CNI at CNCF Amsterdam

Mon, 01 Aug 2022 00:00:00 +0000

On June 22nd, I gave my first meetup talk at the Dutch Kubernetes & Cloud Native meetup in Amsterdam: “CNI from the ground up”.

Why CNI?

Pod networking is something everyone depends on and almost nobody looks at. You install Calico or Cilium, pods get IP addresses, and they can reach each other. It works, so there is no reason to ask how.

That was true for me as well until I read the CNI specification and realized how small it is. There is no daemon, no API server and no gRPC service. A CNI plugin is an executable that the container runtime invokes once per container. The network configuration arrives on stdin, the parameters arrive as environment variables, and the plugin prints a JSON result to stdout. That is the entire interface.

How the runtime invokes the plugin: config on stdin, parameters as environment variables, result on stdout

This makes for a good talk topic: a piece of infrastructure that looks like magic from the outside and turns out to be approachable once you look inside.

toycni

Instead of walking through someone else’s implementation, I wrote my own: toycni, a CNI plugin in about 250 lines of Go.

For the ADD command, it

creates a Linux bridge on the node if it doesn’t exist yet and assigns the first IP of the node’s pod subnet to it,
delegates IP allocation to the host-local IPAM plugin, which is just another binary to execute,
creates a veth pair, moves one end into the container’s network namespace and attaches the other end to the bridge,
assigns the allocated IP to the container interface and sets the bridge as the default gateway,
prints the result as JSON.

DEL releases the IP and relies on the runtime deleting the network namespace, which removes the veth pair with it.

The implementation shells out to ip instead of using netlink. That is not how a production plugin should be written, but it means every step maps to a command the audience can run themselves afterwards.

The demo ran on a two node kubeadm cluster in multipass VMs. Each node got its own pod subnet, a static route to the other node’s subnet and an iptables masquerade rule for traffic leaving the cluster. Those last two pieces are where real CNI plugins spend most of their complexity. toycni gets away with static routes only because the demo cluster has exactly two nodes that never change.

How it went

Around 50 people attended. Speaking with some of them afterwards, I got very positive feedback about both the content and the presentation style, which was a relief for a first talk. Unfortunately the recording failed due to technical issues, so only the slides and the code remain.

Preparing the talk taught me more than giving it did. Writing a plugin that actually has to work in a cluster forces you to understand the parts of the specification you would otherwise skim over. I can recommend it as a way to learn a topic, even without a talk at the end of it.

Networking on Nick's Tech Blog

Talk on CNI at CNCF Amsterdam

Why CNI?

toycni

How it went