Post

Cilium migration trial values prepare for CNI coexistence without Argo CD registration

Cilium migration trial values prepare for CNI coexistence without Argo CD registration

The latest main push adds a Cilium migration trial directory and a cluster-topology note, preparing a chart overlay before any GitOps registration. The change keeps kube-proxy running and leaves Flannel and Multus as the active CNI while Cilium runs in a non-enforcing coexistence mode.

Implementation: Cilium migration trial overlay

I added clusters/homelab/platform/cilium/README.md documenting values-migration.yaml, which renders the initial overlay for the Cilium chart. The directory has no Argo CD registration. During coexistence the values keep kube-proxy running, disable Cilium policy enforcement, and let Flannel and Multus own the active CNI configuration. Talos supplies cgroup v2 and agents run without the kernel-module loading capability. The Helm snippet targets the system namespace and the current Kubernetes version. The cluster-topology.md note adds one line stating that public app address records stay until a later guarded retirement.

Verification: rollout checkpoints and pending deployment checks

The README lists rollout checkpoints: exercise a trial with Flannel, thick Multus, and Istio ambient, then verify cross-overlay Pod and Service connectivity, DNS, and the Octelium secondary attachment as nodes switch over. It also calls for publishing and admitting exact chart images through the Harbor image pipeline, registering the overlay through GitOps, and rehearsing protected node-by-node cutover and rollback while retaining a working route to the Talos and Kubernetes APIs. Because this push only prepares values and does not register the chart, runtime verification is pending; the next check is to confirm the overlay renders cleanly and the CIDR is unused by cluster, LAN, and tailnet routes before any registration.

Platform engineering: CNI coexistence pattern for platform teams

This approach transfers to platform teams running multi-CNI clusters: stage the new CNI values in a separate directory without Argo CD registration, keep the existing CNI active during coexistence, disable policy enforcement on the incoming plugin, and define cutover checkpoints that include rollback and API reachability. A homelab owner can use the same technique to trial Cilium alongside Flannel and Multus before committing to a cluster-wide migration.

This post is licensed under CC BY 4.0 by the author.