CAPA Releases

  • WARNING: This release enables the OwnerReferencesPermissionEnforcement admission controller by default. This means that only users with delete permission to an object can change its metadata.ownerReferences, and only users with update permission to the finalizers subresource of the referenced owner can change metadata.ownerReferences[x].blockOwnerDeletion. If you have workloads that need to modify these fields, please ensure that the necessary RBAC permissions are in place before upgrading to this release.

    Example:

    - apiGroups: ["<group>"]
      resources: ["<resource>", "<resource>/finalizers"]
      verbs: ["delete", "..."]  # Add any additional verbs your use case requires
    

    Changes compared to v32.0.0

    Components

    • cluster-aws from v5.0.0 to v6.0.0
    • Flatcar from v4230.2.2 to v4230.2.3
    • Kubernetes from v1.32.9 to v1.33.5

    cluster-aws v5.0.0…v6.0.0

    Added

    • Add standard tags to IRSA infrastructure.
    • Expose value to configure terminationGracePeriod in the karpenter node pools.

    Changed

    • Chart: Update cluster to v4.2.0.
    • The container registry passed as value to default apps is set to gsoci.azurecr.io, regardless of the cluster region. The mirroring feature of containerd will make sure the right registry is used.
    • Switch to HelmReleases to install karpenter and karpenter-crossplane-resources charts.
    • Bump flux HelmReleases api version to v2.
    • Reduce heartbeat timeout for ASG lifecycle hooks to from 30 minutes to 3 minutes since aws-node-termination-handler-app (NTH) can now send heartbeats
    • Configure the following startupTaints to help karpenter ignore pending Pods due to these taints that will be removed after the node starts, avoiding unnecessary instance provisioning:
      • node.cluster.x-k8s.io/uninitialized:NoSchedule
      • node.cilium.io/agent-not-ready:NoSchedule
      • ebs.csi.aws.com/agent-not-ready:NoExecute
    • Include cilium ENI mode pod CIDRs in the NodePort Services security group ingress rules.

    Removed

    • Removed capi-node-labeler app. From now on, the worker nodes won’t have the node-role.kubernetes.io/worker or node.kubernetes.io/worker labels.

    Apps

    • aws-ebs-csi-driver from v3.0.5 to v3.2.0
    • aws-nth-bundle from v1.2.2 to v1.3.0
    • cert-exporter from v2.9.9 to v2.9.12
    • cert-manager from v3.9.2 to v3.9.3
    • cilium from v1.3.0 to v1.3.1
    • cloud-provider-aws from v1.32.3 to v1.33.2-1
    • cluster-autoscaler from v1.32.2-gs1 to v1.33.1-2
    • coredns from v1.27.0 to v1.28.2
    • etcd-defrag from v1.0.8 to v1.2.1
    • etcd-k8s-res-count-exporter from v1.10.7 to v1.10.9
    • k8s-audit-metrics from v0.10.6 to v0.10.8
    • Added karpenter v1.3.0
    • Added karpenter-crossplane-resources v0.4.0
    • node-exporter from v1.20.5 to v1.20.7
    • observability-bundle from v2.2.2 to v2.3.2
    • vertical-pod-autoscaler from v6.0.1 to v6.1.1
    • vertical-pod-autoscaler-crd from v4.0.1 to v4.1.1

    aws-ebs-csi-driver v3.0.5…v3.2.0

    Changed

    • Configure gsoci.azurecr.io as the default container image registry.
    • Set default updateStrategy.rollingUpdate.maxUnavailable to 25% in DaemonSet to speed up rolling update.

    aws-nth-bundle v1.2.2…v1.3.0

    Changed

    • Upgrade aws-nth-crossplane-resources to v1.3.0, fixing support for multiple OIDC providers in the NTH IAM role as required for cleanup of migrated vintage clusters, and supporting heartbeat sending
    • Upgrade aws-node-termination-handler-app to v1.23.0, enabling heartbeats by default and upgrading to upstream application version v1.25.2 which fixes a resource leak bug relevant to heartbeat sending
    • Upgrade aws-nth-crossplane-resources to v1.1.1, supporting multiple OIDC providers in the NTH IAM role as required for cleanup of migrated vintage clusters

    cert-exporter v2.9.9…v2.9.12

    Changed

    • Go: Update dependencies.
    • Chart: Add value to toggle creation of Daemonset resources.
    • Go: Update dependencies.

    cert-manager v3.9.2…v3.9.3

    Changed

    • Fix missing targetPort in cainjector-service

    cilium v1.3.0…v1.3.1

    Changed

    cloud-provider-aws v1.32.3…v1.33.2-1

    Changed

    • Chart: Update to upstream v1.33.2.

    cluster-autoscaler v1.32.2-gs1…v1.33.1-2

    Changed

    • Chart: Grant access to VolumeAttachments. (#345)
    • Update Kyverno API to v2 for policy exceptions
    • Chart: Update to upstream v1.33.1.

    coredns v1.27.0…v1.28.2

    Changed

    • Update coredns image to 1.13.1.
    • Add value to toggle creation of controlplane deployment.
    • Update coredns image to 1.13.0.

    etcd-defrag v1.0.8…v1.2.1

    Changed

    • Chart: Update dependency ahrtr/etcd-defrag to v0.34.0. (#62)
    • Chart: Update dependency ahrtr/etcd-defrag to v0.33.0. (#60)
    • Update Kyverno API to v2 for policy exceptions
    • Chart: Update dependency ahrtr/etcd-defrag to v0.32.0. (#57)

    etcd-k8s-res-count-exporter v1.10.7…v1.10.9

    Changed

    • Go: Update dependencies.
    • Update Kyverno API to v2 for policy exceptions
    • Go: Update dependencies.

    k8s-audit-metrics v0.10.6…v0.10.8

    Changed

    • Go: Update dependencies.
    • Update Kyverno API to v2 for policy exceptions
    • Go: Update dependencies.

    karpenter v1.3.0

    Changed

    • Updated karpenter to 1.7.1

    karpenter-crossplane-resources v0.4.0

    Changed

    • Add iam:ListInstanceProfiles for release 1.7.1

    node-exporter v1.20.5…v1.20.7

    Changed

    • Go: Update dependencies.
    • Update Kyverno API to v2 for policy exceptions
    • Go: Update dependencies.

    observability-bundle v2.2.2…v2.3.2

    Added

    • Add KSM metrics for cloudnative-pg Cluster objects

    Changed

    • Update alloy-app to 0.15.0
      • Bumps alloy to 1.11.0

    Fixed

    • Update alloy-app to 0.15.1
      • Bumps alloy to 1.11.2

    vertical-pod-autoscaler v6.0.1…v6.1.1

    Changed

    • Chart: Update Helm release vertical-pod-autoscaler to v11.1.1. (#375)
    • Chart: Update Helm release vertical-pod-autoscaler to v11.1.0. (#372)

    vertical-pod-autoscaler-crd v4.0.1…v4.1.1

    Changed

    • Chart: Sync to upstream. (#166)
    • Chart: Sync to upstream. (#164)
  • WARNING: With Flatcar 4230.2.0, cgroups v1 backwards compatibility has been removed. This means that enabling legacy cgroups v1 is no longer supported and nodes still using them will fail to update.

    Changes compared to v31.1.2

    Components

    • cluster-aws from v3.6.2 to v5.0.0
    • Flatcar from v4152.2.3 to v4230.2.2
    • Kubernetes from v1.31.11 to v1.32.9

    cluster-aws v3.6.2…v5.0.0

    Added

    • Add global.connectivity.network.nodePortIngressRuleCidrBlocks value to allow configuring the CIDRs in the NodePort security group ingress rules.
    • Expose new machinepool values to configure the karpenter node pools:
    • consolidateAfter
    • consolidationBudgets
    • consolidationPolicy

    Changed

    • Chart: Update cluster to v3.0.1.
      • BREAKING CHANGE: Cgroups v1 is not supported anymore. The .internal.advancedConfiguration.cgroupsv1 and .global.nodePools.().cgroupsv1 flags have been removed.
      • Chart: Simplify containerd configuration by using a single config file for both control-plane and worker nodes.
    • Chart: Update cluster to v2.6.2.
    • Chart: Update cluster to v2.6.1.

    Fixed

    • Add cluster chart nodepool fields to the schema.

    Removed

    • Remove Helm chart that creates karpenter node pools, because they will be created by a kubernetes controller running in the management cluster.

    Apps

    • capi-node-labeler from v1.1.2 to v1.1.3
    • cert-exporter from v2.9.8 to v2.9.9
    • cert-manager from v3.9.1 to v3.9.2
    • cilium from v1.2.2 to v1.3.0
    • cloud-provider-aws from v1.31.5-gs1 to v1.32.3
    • cluster-autoscaler from v1.31.3-gs1 to v1.32.2-gs1
    • coredns from v1.26.0 to v1.27.0
    • etcd-defrag from v1.0.6 to v1.0.8
    • etcd-k8s-res-count-exporter from v1.10.6 to v1.10.7
    • k8s-audit-metrics from v0.10.5 to v0.10.6
    • k8s-dns-node-cache from v2.9.0 to v2.9.1
    • karpenter-bundle from v2.1.0 to v2.2.0
    • metrics-server from v2.6.0 to v2.7.0
    • node-exporter from v1.20.4 to v1.20.5
    • observability-bundle from v2.0.0 to v2.2.2
    • vertical-pod-autoscaler from v5.5.1 to v6.0.1
    • vertical-pod-autoscaler-crd from v3.3.1 to v4.0.1

    capi-node-labeler v1.1.2…v1.1.3

    Changed

    • Go: Update dependencies.

    cert-exporter v2.9.8…v2.9.9

    Changed

    • Go: Update dependencies.

    cert-manager v3.9.1…v3.9.2

    Changed

    • Add alloy ingress rules for cainjector metrics ingestion.

    cilium v1.2.2…v1.3.0

    Changed

    • Upgrade Cilium to v1.18.1.
    • Improve the k8s service host autodiscovery mechanism
    • Upgrade Cilium to v1.17.7.

    cloud-provider-aws v1.31.5-gs1…v1.32.3

    Changed

    • Chart: Update to upstream v1.32.3.
    • Chart: Update to upstream v1.32.3.

    cluster-autoscaler v1.31.3-gs1…v1.32.2-gs1

    Changed

    • Chart: Update to upstream v1.32.2.

    coredns v1.26.0…v1.27.0

    Changed

    • Updated E2E tests to use apptest-framework v1.14.0
    • Update coredns image to 1.12.3.

    etcd-defrag v1.0.6…v1.0.8

    Changed

    • Chart: Update dependency ahrtr/etcd-defrag to v0.31.0. (#52)
    • Chart: Update dependency ahrtr/etcd-defrag to v0.30.0. (#46)

    etcd-k8s-res-count-exporter v1.10.6…v1.10.7

    Changed

    • Go: Update dependencies.

    k8s-audit-metrics v0.10.5…v0.10.6

    Changed

    • Go: Update dependencies.

    k8s-dns-node-cache v2.9.0…v2.9.1

    Changed

    • Update PolicyException apiVersion to v2.

    karpenter-bundle v2.1.0…v2.2.0

    Changed

    • Change karpenter to it’s own namespace.
    • Bump karpenter to v1.6.3.
    • Allow changing karpenter app versions.

    metrics-server v2.6.0…v2.7.0

    Changed

    • Chart: Update PolicyExceptions to v2.

    node-exporter v1.20.4…v1.20.5

    Changed

    • Go: Update dependencies.

    observability-bundle v2.0.0…v2.2.2

    Added

    • Add KSM metrics for IRSAClaim objects

    Changed

    • Upgrade kube-prometheus-stack-app to 18.1.0
      • Add relabeling rules from cluster-api-monitoring-app so that cluster_id label points to the workload cluster name as expected in some alert definitions
    • Upgrade kube-prometheus-stack to 77.0.1
      • Bumps prometheus-operator and CRDs to 0.85.0
    • Update alloy-app to 0.13.0
    • Upgrade kube-prometheus-stack to 76.4.0
      • Bumps prometheus-operator and CRDs to 0.84.1
      • Bumps prometheus to 3.5.0
    • Update alloy-app to 0.12.1
      • Bumps alloy to 1.10.1

    vertical-pod-autoscaler v5.5.1…v6.0.1

    Changed

    • Chart: Update Helm release vertical-pod-autoscaler to v11.0.1. (#370)
    • Chart: Update Helm release vertical-pod-autoscaler to v11.0.0. (#362)

    vertical-pod-autoscaler-crd v3.3.1…v4.0.1

    Changed

    • Chart: Sync to upstream. (#162)
    • Chart: Sync to upstream. (#154)
  • This release updates the cluster-aws chart to fix an issue around Helm values schema validation when using certain node pool fields.

    Changes compared to v31.1.1

    Components

    • cluster-aws from v3.6.1 to v3.6.2

    cluster-aws v3.6.1…v3.6.2

    Fixed

    • Add cluster chart nodepool fields to the schema.
  • This release updates the cluster-aws chart and the underlying cluster chart to address an issue around Helm values schema validation uncovered by newer Helm versions.

    Changes compared to v31.1.0

    Components

    • cluster-aws from v3.6.0 to v3.6.1

    cluster-aws v3.6.0…v3.6.1

    Changed

    • Chart: Update cluster to v2.5.1.
  • This release updates the cluster-aws chart and the underlying cluster chart to address an issue around Helm values schema validation uncovered by newer Helm versions.

    Changes compared to v30.1.4

    Components

    • cluster-aws from v3.2.3 to v3.2.4

    cluster-aws v3.2.3…v3.2.4

    Changed

    • Chart: Update cluster to v2.2.2.
  • This release backports a fix for reducing IMDS Response Hop Limit to 2 if pod networking is in ENI mode to increase security. For clusters in ENI mode, all nodes are rolled when upgrading to this release.

    Changes compared to v30.1.3

    Components

    • cluster-aws from v3.2.2 to v3.2.3

    cluster-aws v3.2.2…v3.2.3

    Changed

    • Reduce IMDS Response Hop Limit to 2 if pod networking is in ENI mode to increase security.

    Apps

  • This release backports a fix for reducing IMDS Response Hop Limit to 2 if pod networking is in ENI mode to increase security. For clusters in ENI mode, all nodes are rolled when upgrading to this release.

    Changes compared to v29.6.3

    Components

    • cluster-aws from v2.6.3 to v2.6.4

    cluster-aws v2.6.3…v2.6.4

    Changed

    • Reduce IMDS Response Hop Limit to 2 if pod networking is in ENI mode to increase security.

    Apps

  • This release updates Kubernetes to the latest patch release v1.31.11.

    During control plane upgrades, short-term warnings are now prevented by setting a fixed instead of dynamic AMI lookup string. This leads to nodes being rolled once when upgrading to this release.

    We added an option to set the IMDSv2 request hop limit for EC2 instances ‒ this is usually not needed, except if security requirements such as AWS SCPs (service control policies) dictate a maximum.

    Karpenter support keeps getting better: node-termination-handler is not installed anymore if only Karpenter node pools are used, as the same function is built into Karpenter (pod draining and EC2 instance termination handling). Nodes in such pools now also use the reduced IAM permission set (can be toggled in exceptional cases).

    Changes compared to v31.0.0

    Components

    • cluster-aws from v3.4.0 to v3.6.0
    • Kubernetes from v1.31.9 to v1.31.11

    cluster-aws v3.4.0…v3.6.0

    Added

    • Add giantswarm.io/role: nodes by default to private subnets used for nodes. Can be overwritten.
    • Make IMDSv2 hop limit configurable

    Changed

    • Chart: Update cluster to v2.5.0.
    • Only deploy node-termination-handler when there are non-karpenter node pools because karpenter takes care of node draining
    • Change imageLookupFormat to use a static string rather than CAPI replacing the OS and Kubernetes versions. This rolls control plane nodes.

    Fixed

    • Use reduced IAM permissions on karpenter worker nodes instance profile. This can be toggled back with global.providerSpecific.reducedInstanceProfileIamPermissionsForWorkers.

    Apps

    • aws-nth-bundle from v1.2.1 to v1.2.2
    • capi-node-labeler from v1.1.1 to v1.1.2
    • cert-exporter from v2.9.7 to v2.9.8
    • cilium from v1.2.1 to v1.2.2
    • cluster-autoscaler from v1.31.2-gs2 to v1.31.3-gs1
    • coredns from v1.25.0 to v1.26.0
    • etcd-defrag from v1.0.5 to v1.0.6
    • etcd-k8s-res-count-exporter from v1.10.5 to v1.10.6
    • k8s-audit-metrics from v0.10.4 to v0.10.5
    • k8s-dns-node-cache from v2.8.1 to v2.9.0
    • karpenter-bundle from v2.0.0 to v2.1.0
    • karpenter-nodepools from v0.1.0 to v0.2.0
    • node-exporter from v1.20.3 to v1.20.4
    • security-bundle from v1.11.0 to v1.12.0
    • teleport-kube-agent from v0.10.5 to v0.10.6

    aws-nth-bundle v1.2.1…v1.2.2

    Changed

    • Upgrade Node Termination Handler to 1.21.0.

    capi-node-labeler v1.1.1…v1.1.2

    Changed

    • Go: Update dependencies.

    cert-exporter v2.9.7…v2.9.8

    Changed

    • Go: Update dependencies.

    cilium v1.2.1…v1.2.2

    Changed

    • Upgrade Cilium to v1.17.6.
    • Updated E2E tests to use apptest-framework v1.14.0
    • Increase Cilium operator resource limits.

    Removed

    • Remove deprecated “partial” mode from Kube Proxy Replacement options.

    cluster-autoscaler v1.31.2-gs2…v1.31.3-gs1

    Changed

    • Chart: Update to upstream v1.31.3.

    coredns v1.25.0…v1.26.0

    Changed

    • Update coredns image to 1.12.2.

    etcd-defrag v1.0.5…v1.0.6

    Changed

    • Chart: Update dependency ahrtr/etcd-defrag to v0.29.0. (#43)

    etcd-k8s-res-count-exporter v1.10.5…v1.10.6

    Changed

    • Go: Update dependencies.

    k8s-audit-metrics v0.10.4…v0.10.5

    Changed

    • Go: Update dependencies.

    k8s-dns-node-cache v2.8.1…v2.9.0

    Changed

    • Upgrade application to version 1.26.4 (includes coredns 1.11.3)
    • Increase ServiceMonitor’s scrapping interval to 1m.
    • Remove obsolete PSPs

    karpenter-bundle v2.0.0…v2.1.0

    Removed

    • Remove capa-karpenter-taint-remover because nodes are now in the MachinePool CR, so the taint will be removed by CAPI.

    karpenter-nodepools v0.1.0…v0.2.0

    Changed

    • Improve json schema.
    • Change subnet selector to avoid CNI subnets.

    node-exporter v1.20.3…v1.20.4

    Changed

    • Go: Update to v1.24.5.

    security-bundle v1.11.0…v1.12.0

    Changed

    • Update trivy-operator (app) to v0.11.1.
    • Update trivy (app) to v0.14.0.
    • Update falco (app) to v0.10.1.
    • Update cloudnative-pg (app) to v0.0.10.
    • Update starboard-exporter (app) to v0.8.2.
    • Updated E2E tests to use apptest-framework v1.14.0

    teleport-kube-agent v0.10.5…v0.10.6

    Changed

    • AppVersion upgrade to 17.5.4
  • Upgrade cluster-aws to handle IMDS Hop Limit and patch kubernetes version.

    Changes compared to v28.5.4

    Components

    • cluster-aws from v1.3.10 to v1.3.11

    cluster-aws v1.3.10…v1.3.11

    Changed

    • Reduce IMDS Response Hop Limit to 2 if pod networking is in ENI mode to increase security.
  • Upgrade cluster-aws to handle IMDS Hop Limit.

    Changes compared to v27.5.3

    Components

    • cluster-aws from v1.3.10 to v1.3.11

    cluster-aws v1.3.10…v1.3.11

    Changed

    • Reduce IMDS Response Hop Limit to 2 if pod networking is in ENI mode to increase security.