Tenant Cluster Releases for KVM

  • Nodes will be rolled during upgrade to this version.

    This patch release adds registry credentials to prevent an issue with image pulling rate limits recently introduced by Docker Hub.

    Note before upgrade:

    Please contact your Solution Engineer before upgrading. The upgrade is automated. However, it includes a data migration from Helm 2 release configmaps to Helm 3 release secrets, there are some pre-upgrade checks and we recommend monitoring the upgrade to ensure safety.

    Note for Solution Engineers:

    Before upgrading, please ensure cluster is on KVM 12.1.x platform release first.

    Please use Upgrading tenant clusters to Helm 3 as a guide on the upgrade process for the checks and monitoring steps.

    Note for future 12.x.x releases:

    Please ensure cluster is on KVM 12.1.x platform release first before upgrading to 12.2.0+ Please persist this note until all customers are on KVM 12.1.x and above.

    Change details

    kvm-operator v3.13.0

    • Update k8scloudconfig to v7.2.0, containing a fix for DockerHub QPS.
  • If you are upgrading from 12.3.0, upgrading to this release will not roll your nodes.

    This patch release fixes a problem causing the accidental deletion and reinstallation of Preinstalled Apps (such as CoreDNS) in 12.x.x tenant clusters.

    Please upgrade all older clusters to this version in order to prevent possible downtime.

    Note before upgrade:

    Please contact your Solution Engineer before upgrading. The upgrade is automated. However, it includes a data migration from Helm 2 release configmaps to Helm 3 release secrets, there are some pre-upgrade checks and we recommend monitoring the upgrade to ensure safety.

    Note for Solution Engineers:

    Before upgrading, please ensure cluster is on KVM 12.1.x platform release first.

    Please use Upgrading tenant clusters to Helm 3 as a guide on the upgrade process for the checks and monitoring steps.

    Note for future 12.x.x releases:

    Please ensure cluster is on KVM 12.1.x platform release first before upgrading to 12.2.0+ Please persist this note until all customers are on KVM 12.1.x and above.

    Change details

    cluster-operator 0.23.18

    • Remove all chartconfig migration logic that caused accidental deletion and is no longer needed.

    app-operator 2.3.5

    • Fix YAML comparison for chart configmaps and secrets.
  • This release upgrades all Helm releases managed by Giant Swarm to use Helm v3.3.4.

    This lets us benefit from the improved security model and keep up to date with the community. We also remove the Tiller deployment from the giantswarm namespace, removing its gRPC endpoint, which reduces operational complexity.

    If you are still using Helm 2 then these Helm releases will not be affected. However we encourage you to upgrade to Helm 3 as Helm 2 support ends on November 13th 2020. https://helm.sh/blog/helm-v2-deprecation-timeline/

    The release also upgrades Container Linux to address security vulnerabilities.

    Below, you can find more details on components that were changed with this release.

    Note before upgrade:

    Please contact your Solution Engineer before upgrading. The upgrade is automated. However, it includes a data migration from Helm 2 release configmaps to Helm 3 release secrets, there are some pre-upgrade checks and we recommend monitoring the upgrade to ensure safety.

    Note for Solution Engineers:

    Before upgrading, please ensure cluster is on KVM 12.1.x platform release first.

    Please use Upgrading tenant clusters to Helm 3 as a guide on the upgrade process for the checks and monitoring steps.

    Note for future 12.x.x releases:

    Please ensure cluster is on KVM 12.1.x platform release first before upgrading to 12.2.0+ Please persist this note until all customers are on KVM 12.1.x and above.

    Change details

    app-operator v2.3.4

    chart-operator v2.3.5

    kvm-operator v3.12.2

    • Added monitoring labels into prometheus metrics

    containerlinux 2512.5.0

    Security fixes:

    Changes:

    • Update public key to include a new subkey
    • Vultr support in Ignition (flatcar-linux/ignition#13)
    • VMware OVF settings default to ESXi 6.5 and Linux 3.x

    Updates:

  • If you are upgrading from 12.1.0, upgrading to this release will not roll your nodes.

    As of this release, NGINX Ingress Controller App is now an optional (and not pre-installed) component on KVM.

    This enables use of alternative ingress controllers without wasting resources where NGINX is not the preferred option.

    Now NGINX App installations can be managed and updated independent of the cluster, which is both a benefit and a new responsibility 😅

    Upgrading tenant clusters with pre-installed NGINX will leave NGINX unchanged. Existing NGINX App custom resources will still have giantswarm.io/managed-by: cluster-operator label, but it will be ignored. The label will be cleaned up later after all tenant clusters have been upgraded and KVM platform releases older than v12.2.0 archived.

    Note for cluster upgrades:

    Please ensure cluster is on KVM 12.1.x platform release first before upgrading the cluster to 12.2.0+

    Below, you can find more details on components that were changed with this release.

    cluster-operator 0.23.14

    • Support for making NGINX IC App optional and not pre-installed.
  • This release includes two significant improvements to NGINX Ingress Controller. It also includes a fix for Quay being a single point of failure by using Docker mirroring feature. This ensures availability of all images needed for node bootstrap, thus the cluster creation/scaling doesn’t depend on Quay availability anymore.

    The two NGINX Ingress Controller improvements:

    • Multiple NGINX Ingress Controllers per tenant cluster are now supported, enabling separation of internal vs external traffic, dev vs prod, and so on.
    • Management of NGINX IC NodePort Service is moved from kvm-operator to NGINX IC App itself. This enables configurability of external traffic policy and lays the foundation for making NGINX IC App optional and not pre-installed in a future KVM platform release.

    Along with kvm-operator, cluster-operator and NGINX IC, release includes several upstream component upgrades.

    Note for cluster upgrades:

    Please manually delete nginx-ingress-controller NodePort Service in kube-system namespace. Upgrading the cluster then recreates the NodePort Service and moves its management from kvm-operator to NGINX IC. To minimize downtime, please delegate cluster upgrades to your SE.**

    Note for future 12.1.x releases:

    To prevent downtime, please persist this and the above note until all customers are on 12.1.0 and above.

    Below, you can find more details on components that were changed with this release.

    cluster-operator 0.23.13

    • Enable NGINX App managed NodePort Service on KVM.

    kube-state-metrics v1.9.7 (Giant Swarm app v1.1.1)

    • Updated kube-state-metrics version from 1.9.5 to 1.9.7. Check the upstream changelog for details on all changes.

    kvm-operator v3.12.1

    • Add registry mirrors support.
    • Stop provisioning NGINX IC NodePort Service.

    metrics-server v0.3.6 (Giant Swarm app v1.1.1)

    • Updated metrics-server version from 0.3.3 to 0.3.6. Check the upstream changelog for details on all changes.

    nginx-ingress-controller v0.34.1 (Giant Swarm app v1.8.1)

    • Support multiple NGINX IC App installations per tenant cluster.
    • Made NGINX NodePort Service external traffic policy configurable.
    • Made NGINX NodePort Service node ports configurable.
    • Drop support for deprecated configuration properties.

    node-exporter v1.0.1 (Giant Swarm app v1.3.0)

    • Updated node-exporter version from 0.18.1 to 1.0.1. Check the upstream changelog for details on all changes.
  • If you are upgrading from 12.0.0, upgrading to this release will not roll your nodes. It will only update the apps.

    This release updates NGINX Ingress Controller to the latest upstream release. Most importantly, it includes a fix for a regression introduced in the previous upstream release related to use-regex and rewrite annotations.

    Below, you can find more details on components that were changed with this release.


    nginx-ingress-controller 1.7.3

    • Upgraded upstream ingress-nginx from 0.34.0 to 0.34.1 - changelog.
  • This is the first Giant Swarm release which includes Kubernetes v1.17. Many core components and default apps have been updated for improved reliability and observability. Further details about changes to individual components can be found below.


    Kubernetes 1.17.8

    • Cloud Provider Labels reach General Availability: Added as a beta feature way back in v1.2, v1.17 sees the general availability of cloud provider labels.
    • Volume Snapshot Moves to Beta: The Kubernetes Volume Snapshot feature is now beta in Kubernetes v1.17. It was introduced as alpha in Kubernetes v1.12, with a second alpha with breaking changes in Kubernetes v1.13.
    • CSI Migration Beta: The Kubernetes in-tree storage plugin to Container Storage Interface (CSI) migration infrastructure is now beta in Kubernetes v1.17. CSI migration was introduced as alpha in Kubernetes v1.14.

    Calico 3.14.1

    • Added port 6443 (Kubernetes API server) to failsafe ports for felix.
    • Fixed IPv6 rogue router advertisement vulnerability (CVE-2020-13597).

    etcd 3.4.9

    kvm-operator 3.12.0

    • Improved upgrades from KVM v11 releases.
    • Fixed CR validation errors during cluster creation.
    • Upgraded node image to QEMU 4.2.0 and Fedora 32.
    • Modified Calico deployment to use -bird-live as a liveness probe improving observability of failed mesh networking.
    • Removed limits from calico-kube-controllers to prevent it from being OOM killed.

    cluster-operator 0.23.12

    • Aligned with NGINX IC App 1.7.0 moving LB Service management from the operator to the app itself.

    Container Linux 2512.2.1

    cert-exporter 1.2.3

    • Updated prometheus/client_golang dependency.
    • Changed to App-based deployment.

    CoreDNS 1.2.0

    • Made resource requests/limits configurable.
    • Make forward options optional.
    • Apply a readiness probe.
    • Increase the liveness probe failure threshold from 5 failures to 7 failures.

    kube-state-metrics 1.1.0

    • Added 100.64.0.0/10 to the allowed egress subnets in NetworkPolicy.
    • Fixed invalid cluster role binding for Helm 3 compatibility.

    metrics-server 1.1.0

    • Added 100.64.0.0/10 to the allowed egress subnets in NetworkPolicy.

    net-exporter 1.9.0

    • Added ntp collector.

    nginx-ingress-controller 1.7.2

    • Upgraded upstream ingress-nginx from 0.30.0 to 0.34.0.
    • Improved observability, enabled monitoring Service by default for monitoring targets discovery and removed support for disabling it.
    • Added support for additional service for internal traffic. Existing service can still be configured to be either for public (default) or internal traffic.
    • Made monitoring headless Service non-optional.
    • Enabled managed app monitoring via monitoring service.

    node-exporter 1.2.0

    • Changed Priority Class to system-node-critical.
  • If you are upgrading from 11.3.1, upgrading to this release will not roll your nodes. It will only update the apps.

    This release improves the reliability of NGINX Ingress Controller.

    Specifically, liveness probe is configured to be more fault tolerant than readiness probe. This helps shed load when it goes beyond replica capacity, speeding up recovery when NGINX gets overloaded.

    Below, you can find more details on components that were changed with this release.

    nginx-ingress-controller v0.30.0 (Giant Swarm app v1.6.12)

    • Made healthcheck probes configurable.
    • Made liveness probe more resilient.
    • Aligned labels using app.kubernetes.io/name instead of k8s-app where possible. k8s-app remains to be used for compatibility reasons as selectors are not modifiable without recreating the Deployment.
  • This release fixes a rare bug that would prevent the NGINX IC from being installed on a new cluster.

    This bug would only occur on cluster creation if you had a nginx-ingress-controller-user-values configmap in the kube-system namespace while the cluster was still initialising.

    Solution Engineers have already done the manual fix for affected customers.

    cluster-operator v0.23.9

    • Fix a bug in user values migration logic for apps.
  • This release fixes a problem that prevented clusters with OIDC user and group prefix settings to work as expected in 11.3.0.

    kvm-operator v3.11.1

    • Use Release.Revision in Helm chart for Helm 3 support.
    • Fix OIDC settings which are passed to masters.