Edit

Share via

Update Azure CNI IPAM mode and data plane technology for Azure Kubernetes Service (AKS) clusters

Existing Azure Kubernetes Service (AKS) clusters inevitably need an update to newer IP assignment management (IPAM) modes and data plane technologies to access the latest features and supportability. This article provides guidance on updating an existing AKS cluster to use Azure CNI Overlay for the IPAM mode and Azure CNI Powered by Cilium as the data plane.

Update the IPAM mode to Azure CNI Overlay

Updating an existing cluster to Azure CNI Overlay is an irreversible process.

You can update an existing AKS cluster to Azure CNI Overlay if the cluster:

  • Is on Kubernetes version 1.27 or later.
  • Doesn't use the dynamic IP allocation feature.
  • Doesn't have network policies enabled. If you need to uninstall the network policy engine before updating your cluster, follow the steps in Uninstall Azure Network Policy Manager or Calico.
  • Doesn't use any Windows node pools with Docker as the container runtime.

Before Windows OS build 20348.1668, there was a limitation around Windows overlay pods incorrectly routing packets from host network pods via Source Network Address Translation (SNAT). This limitation had a detrimental effect for clusters that were updating to Azure CNI Overlay. To avoid this issue, use Windows OS build 20348.1668 or later.

Warning

  • If you're using a custom azure-ip-masq-agent configuration to include additional IP ranges that shouldn't send SNAT packets from pods, updating to Azure CNI Overlay can break connectivity to these ranges. Pod IPs from the overlay space are unreachable by anything outside the cluster nodes.

  • For old clusters, a ConfigMap might be left over from a previous version of azure-ip-masq-agent. If this ConfigMap (named azure-ip-masq-agent-config) exists and isn't intentionally in place, you should delete it before updating.

  • If you're not using a custom ip-masq-agent configuration, only the azure-ip-masq-agent-config-reconciled ConfigMap should exist with respect to Azure ip-masq-agent ConfigMap. It's updated automatically during the update process.

The update process triggers node pools to be reimaged simultaneously. Updating each node pool separately to Azure CNI Overlay isn't supported. Any disruptions to cluster networking are similar to a node image update or Kubernetes version upgrade where each node in a node pool is reimaged.

Update an existing Azure Container Networking Interface (CNI) cluster to use Azure CNI Overlay by using the az aks update command.

az aks update \
  --name $CLUSTER_NAME \
  --resource-group $RESOURCE_GROUP \
  --network-plugin-mode overlay \
  --pod-cidr 192.168.0.0/16

The --pod-cidr parameter is required when you update from legacy CNI plugins because the pods need to get IPs from a new overlay space. The new overlay space doesn't overlap with the existing Azure CNI Node Subnet plugin.

Classless Inter-Domain Routing (CIDR) for the pod also can't overlap with any virtual network address of the node pools. For example, if your virtual network address is 10.0.0.0/8, and your nodes are in the subnet 10.240.0.0/16, the --pod-cidr parameter can't overlap with 10.0.0.0/8 or the existing service CIDR on the cluster.

Update the data plane to Azure CNI Powered by Cilium

When you enable Cilium in a cluster that uses a different network policy engine (Azure Network Policy Manager or Calico), the network policy engine is uninstalled and replaced with Cilium. For more information, see Uninstall Azure Network Policy Manager or Calico.

You can update an existing cluster to Azure CNI Powered by Cilium if the cluster doesn't have any Windows node pools.

Warning

The update process triggers node pools to be reimaged simultaneously. Updating each node pool separately isn't supported. Any disruptions to cluster networking are similar to a node image update or Kubernetes version upgrade where each node in a node pool is reimaged. Cilium begins enforcing network policies only after all nodes are reimaged.

To perform the update, you need Azure CLI version 2.52.0 or later. Run az --version to see the currently installed version. If you need to install or upgrade, see Install the Azure CLI.

Update an existing cluster to Azure CNI Powered by Cilium by using the az aks update command.

az aks update \
  --name $CLUSTER_NAME \
  --resource-group $RESOURCE_GROUP \
  --network-dataplane cilium
  • Last updated on