Workload Cluster Releases for Azure

• Dec 12, 2024 Workload cluster releases for Azure releases azure-29.4.0

  Changes compared to v29.3.0

  Components

  • cluster-azure from v1.4.0 to v1.5.0
  • Kubernetes from v1.29.10 to v1.29.12

  cluster-azure v1.4.0…v1.5.0

  Changed

  • Chart: Update cluster to v1.7.0.
    • Add teleport-init systemd unit to handle initial token setup before teleport service starts
    • Improve teleport service reliability by adding proper file and service dependencies and pre-start checks

  Apps

  • cert-manager from v3.8.1 to v3.8.2
  • coredns from v1.22.0 to v1.23.0
  • observability-bundle from v1.8.0 to v1.9.0

  cert-manager v3.8.1…v3.8.2

  Changed

  • Changed ownership to team Shield

  Removed

  • Get rid of label giantswarm.io/monitoring_basic_sli as this slo generation label is not used anymore.

  coredns v1.22.0…v1.23.0

  Changed

  • Update coredns image to 1.11.4.
  • Explicitly expose liveness and readiness probe ports in deployments.

  Removed

  • Remove PodSecurityPolicy and associated Resources and values.

  observability-bundle v1.8.0…v1.9.0

  Added

  • Add alloy v0.7.0 as alloyEvents.

  Changed

  • Upgrade alloy-logs and alloy-metrics to chart 0.7.0.
    • Bumps alloy from 1.4.2 to 1.5.0
  • upgrade kube-prometheus-stack from 65.1.1 to 66.2.1
    • prometheus-operator CRDs from 0.75.0 to 0.78.1
    • prometheus-operator from 0.77.1 to 0.78.1
    • prometheus from 2.54.1 to 2.55.1
    • kube-state-metrics from 2.13.0 to 2.14.0
    • grafana from 8.5.0 to 8.6.0
• Nov 13, 2024 Workload cluster releases for Azure releases azure-29.3.0

  Changes compared to v29.2.0

  Components

  • cluster-azure from v1.3.0 to v1.4.0
  • Flatcar from v3975.2.1 to v3975.2.2
  • Kubernetes from v1.29.9 to v1.29.10

  cluster-azure v1.3.0…v1.4.0

  Changed

  • Make external-dns-private app depend on the prometheus-operator-crd app, because it uses ServiceMonitors.

  Apps

  • cert-exporter from v2.9.2 to v2.9.3
  • observability-bundle from v1.6.2 to v1.8.0

  cert-exporter v2.9.2…v2.9.3

  Changed

  • Chart: Enable global.podSecurityStandards.enforced. (#420)

  observability-bundle v1.6.2…v1.8.0

  Changed

  • Upgrade prometheus-agent from v0.6.9 to v0.7.0.
    • Adds extraArgs to be able to use nice features like wal truncation
  • upgrade kube-prometheus-stack from 61.0.0 to 65.1.1
    • prometheus-operator CRDs from 0.73.0 to 0.75.0
    • prometheus-operator from 0.75.0 to 0.77.1
    • prometheus upgraded from 2.53.0 to 2.54.1
    • grafana from 8.2.0 to 8.5.0
    • thanos ruler upgraded from 0.35.1 to 0.36.1
    • prometheus-node-exporter upgraded from 1.8.1 to 1.8.2
  • Add missing depends on annotation on alloy-metrics and alloy-logs to make sure they are deployed after the prometheus-operator-crds.
  • Upgrade alloyLogs to v0.6.1
    • Allow passing PodLogs via helm chart values
    • Upgrade to Alloy v1.4.2 which fixes a bug with component reload/evaluation and keeping Alloy up-to-date
    • Fixes an issue with CiliumNetworkPolicy preventing Alloy to run in clustering mode
• Sep 30, 2024 Workload cluster releases for Azure releases azure-29.2.0

  Changes compared to v29.1.0

  This release does not contain any changes to components or apps, but makes use of an updated machine image, which includes a fix for accessing private Elastic Container Registries (ECR).

• Sep 24, 2024 Workload cluster releases for Azure releases azure-29.1.0

  Changes compared to v29.0.0

  Components

  • cluster-azure from v1.0.0 to v1.3.0
  • Flatcar from v3815.2.5 to v3975.2.1
  • Kubernetes from v1.29.7 to v1.29.9

  cluster-azure v1.0.0…v1.3.0

  Changed

  • Chart: Update cluster to v1.4.1
    • Allow to enable auditd service through global.components.auditd.enabled.
    • Allow configuring kube-controller-manager --node-cidr-mask-size flag.
    • Set MachineDeployment Kubernetes version from release
  • Apps: Use catalog from Release CR.

  Removed

  • Remove Cilium deprecated values.
  • Remove unused internal values from values.schema.json.

  Apps

  • cert-exporter from v2.9.1 to v2.9.2
  • coredns from v1.21.0 to v1.22.0
  • node-exporter from v1.19.0 to v1.20.0
  • observability-bundle from v1.5.3 to v1.6.2
  • security-bundle from v1.8.0 to v1.8.2
  • teleport-kube-agent from v0.9.2 to v0.10.3
  • vertical-pod-autoscaler from v5.2.4 to v5.3.0
  • vertical-pod-autoscaler-crd from v3.1.0 to v3.1.1

  cert-exporter v2.9.1…v2.9.2

  Added

  • Chart: Add VPA and resources configuration for deployment and daemonset. (#382)

  coredns v1.21.0…v1.22.0

  Changed

  • Update coredns image to 1.11.3.

  Removed

  • Removed legacy Giant Swarm monitoring labels as coredns is monitored through a prometheus-operator generated servicemonitor.

  node-exporter v1.19.0…v1.20.0

  Changed

  • Synced with upstream chart v4.38.0 (node-exporter 1.8.2).

  observability-bundle v1.5.3…v1.6.2

  Added

  • Add alloy v0.4.0 as alloyMetrics.

  Changed

  • Fixed alloyMetrics catalog
  • Disable usage reporting to GrafanaLabs by:
    • Bumping alloyLogs and alloyMetrics to v0.4.1.
    • Bumping grafanaAgent to v0.4.6.

  security-bundle v1.8.0…v1.8.2

  Changed

  • Update cloudnative-pg (app) to v0.0.6.
  • Update trivy-operator (app) to v0.10.0.
  • Update kyverno-policy-operator (app) to v0.0.8.
  • Update kyverno (app) to v0.17.16.

  teleport-kube-agent v0.9.2…v0.10.3

  Changed

  • Disable JAMF components on chart templates
  • Fix issues with templates
  • Change ownership to Team Shield
  • Added small fix on podSecurityContext for seccompProfile.
  • Upgraded to Teleport version 16

  vertical-pod-autoscaler v5.2.4…v5.3.0

  Changed

  • Chart: Update Helm release vertical-pod-autoscaler to v9.9.0. (#314)
  • Chart: Consume global.imageRegistry. (#315)

  Removed

  • Chart: Do not override crds.image.tag. (#316)

  vertical-pod-autoscaler-crd v3.1.0…v3.1.1

  Changed

  • Chart: Improve Chart.yaml. (#110)
  • Repository: Some chores. (#111)
• Aug 13, 2024 Workload cluster releases for Azure releases azure-29.0.0

  Changes compared to v28.0.0

  Components

  • Kubernetes from v1.28.12 to v1.29.7

  Apps

  • azure-cloud-controller-manager from v1.28.10-gs1 to v1.29.8-gs1
  • azure-cloud-node-manager from v1.28.10-gs1 to v1.29.8-gs1

  azure-cloud-controller-manager v1.28.10-gs1…v1.29.8-gs1

  Changed

  • Chart: Update to upstream v1.29.8. (#83)

  azure-cloud-node-manager v1.28.10-gs1…v1.29.8-gs1

  Changed

  • Chart: Update to upstream v1.29.8. (#72)
• Aug 12, 2024 Workload cluster releases for Azure releases azure-28.0.0

  Changes compared to v27.0.0

  Components

  • Kubernetes from v1.27.16 to v1.28.12

  Apps

  • azure-cloud-controller-manager from v1.27.18-gs1 to v1.28.10-gs1
  • azure-cloud-node-manager from v1.27.18-gs1 to v1.28.10-gs1

  azure-cloud-controller-manager v1.27.18-gs1…v1.28.10-gs1

  Changed

  • Chart: Update to upstream v1.28.10. (#82)

  azure-cloud-node-manager v1.27.18-gs1…v1.28.10-gs1

  Changed

  • Chart: Update to upstream v1.28.10. (#71)
• Aug 12, 2024 Workload cluster releases for Azure releases azure-27.0.0

  Changes compared to v26.0.0

  Components

  • cluster-azure from v0.18.0 to v1.0.0
  • Flatcar from v3815.2.4 to v3815.2.5
  • Kubernetes from v1.26.15 to v1.27.16

  cluster-azure v0.18.0…v1.0.0

  Changed

  • Chart: Update cluster to v1.1.0. (#325)
    • Machine Template: Adapt new image format.
    • Apps: Enable observability-policies.

  Apps

  • azure-cloud-controller-manager from v1.26.22-gs2 to v1.27.18-gs1
  • azure-cloud-node-manager from v1.26.22-gs2 to v1.27.18-gs1
  • cert-exporter from v2.9.0 to v2.9.1
  • cert-manager from v3.7.6 to v3.8.1
  • k8s-audit-metrics from v0.9.0 to v0.10.0
  • k8s-dns-node-cache from v2.6.2 to v2.8.1
  • net-exporter from v1.19.0 to v1.21.0
  • observability-bundle from v1.3.4 to v1.5.3
  • observability-policies v0.0.1
  • security-bundle from v1.7.1 to v1.8.0
  • teleport-kube-agent from v0.9.0 to v0.9.2
  • vertical-pod-autoscaler from v5.2.2 to v5.2.4

  azure-cloud-controller-manager v1.26.22-gs2…v1.27.18-gs1

  Changed

  • Chart: Update to upstream v1.27.18. (#81)

  azure-cloud-node-manager v1.26.22-gs2…v1.27.18-gs1

  Changed

  • Chart: Update to upstream v1.27.18. (#70)

  cert-exporter v2.9.0…v2.9.1

  Changed

  • Chart: Update PolicyExceptions to v2beta1. (#358)

  cert-manager v3.7.6…v3.8.1

  Added

  • Improves container security by setting runAsGroup and runAsUser greater than zero for all deployments.

  Changed

  • Bump architect-orb@5.3.1 to fix CVE-2024-24790.
  • Improves cainjector’s Vertical Pod Autoscaler
  • Remove quotes from acme-http01-solver-image argument. The quotes are used when looking up the image which causes an error.
  • Changed the way registry is being parsed in helm templates
  • Enable VPA by default

  k8s-audit-metrics v0.9.0…v0.10.0

  Changed

  • Add securityContext.readOnlyRootFilesystem helm value (default true).

  k8s-dns-node-cache v2.6.2…v2.8.1

  Changed

  • Make the app visible for all providers.
  • Reduce security exceptions #89.
    • Enable readOnly FS moving config to emptyDir volume.
    • Remove NET_ADMIN and drop ALL capabilities.
    • Add NET_BIND_SERVICE capability.
    • Add policy exception for require-non-root-groups/autogen-check-runasgroup.
    • Remove disallow-capabilities-* policy exceptions.
  • Update PolicyException CR version to v2beta1.

  net-exporter v1.19.0…v1.21.0

  Changed

  • Enable readOnlyRootFilesystem in securityContext (#376)[https://github.com/giantswarm/net-exporter/pull/376].
  • Update module google.golang.org/grpc to v1.65.0 (#373).
  • Update k8s modules to v0.30.2 (#375).
  • Update quay.io/giantswarm/alpine Docker tag to v3.20.1 (#372).
  • Add node and app labels in ServiceMonitor.

  observability-bundle v1.3.4…v1.5.3

  Added

  • Add alloy v0.3.0 as alloy-logs

  Changed

  • Rename alloy-logs app to camel case alloyLogs.
  • Fix CNP issues (allow traffic from pods in kube-system to nginx-ingress-controller)
    • Upgrade grafana-agent to 0.4.5.
    • Upgrade alloy to 0.3.1.
    • Upgrade promtail to 1.5.4.
  • Upgrade prometheus-operator-crd to 11.0.1.
  • prometheus-operator will not check promql syntax for prometheusRules that are labelled application.giantswarm.io/prometheus-rule-kind: loki
  • Upgrade kube-prometheus-stack to 11.0.0 and prometheus-operator-crd to 11.0.0. This upgrade mainly consists in:
    • kube-prometheus-stack dependency chart upgraded from 56.21.2 to 61.0.0
    • prometheus upgrade from 2.50.1 to 2.53.0
    • thanos ruler upgrade from 0.34.1 to 0.35.1
    • kube-state-metrics from 2.10.0 to 2.12.0
    • prometheus-operator from 0.71.2 0.75.0 - adding remoteWrite.proxyFromEnvironment and Scrape Class support
    • prometheus-node-exporter upgraded from 1.8.0 to 1.8.1
  • Upgrade grafana-agent from 0.4.3 to 0.4.4
    • This version enables the override the grafana agent CiliumNetworkPolicy egress and ingress sections.

  observability-policies v0.0.1

  Added

  • Add a ClusterPolicy to prevent prometheus-operator CRDs deletion.
  • Create observability-policies app to deploy Kyverno Observability Policies into clusters.

  security-bundle v1.7.1…v1.8.0

  Added

  • Add kyverno-crds app to handle Kyverno CRD install.

  Changed

  • Update kyverno (app) to v0.17.15. This version disables the CRD install job in favor of kyverno-crds App.

  teleport-kube-agent v0.9.0…v0.9.2

  Changed

  • Introduced podAntiAffinity so teleport-kube-agent pods run on different control-plane nodes also increased the number of replicas to 3 to maintain better high availability.
  • Changed the way registry is being parsed in helm templates

  vertical-pod-autoscaler v5.2.2…v5.2.4

  Changed

  • Chart: Update Helm release vertical-pod-autoscaler to v9.8.3. (#301)
  • Chart: Change restartPolicy to OnFailure for the CRD job. (#298)
• Jun 27, 2024 Workload cluster releases for Azure releases azure-25.0.0

  We are happy to announce the first release for Azure that uses the new release framework.

  Migration to new releases flow

  In order to consume the new flow, the following two fields need to be manually adapted:

  • In ConfigMap <cluster name>-userconfig set .Values.global.release.version to the release version, e.g. 25.0.0.
  • In App <cluster name> remove the spec.version field. In case of GitOps, Flux might complain that the app manifest is invalid as the spec.version field is mandatory. In that case, edit the live App CR and set spec.version to an empty string. That will unblock Flux and allow it reconcile successfully.

  And if you want to use kubectl-gs to create a cluster, you’d need to now specify the release version, e.g.:

  kubectl-gs template cluster --provider capz --organization my-org --name cluster_name --region westeurope --azure-subscription-id AZURE_ID --release 25.0.0
  

• Jun 27, 2024 Workload cluster releases for Azure releases azure-26.0.0

  Changes compared to v25.0.0

  • Kubernetes version change from 1.25.16 to 1.26.15
  • azure-cloud-controller-manager from 1.24.18-gs6 to 1.26.22-gs2
  • azure-cloud-node-manager from 1.24.18-gs6 to 1.26.22-gs2
  • cluster-azure from 0.17.0 to 0.18.0
  • azuredisk-csi-driver from 1.26.2-gs6 to 1.30.2-gs2.