1. Docs
  2. Changes and releases
  3. Observability

Observability

  • Observability mimir-app v0.30.1

    Added

    • Allow InPlaceOrRecreate and InPlace as verticalAutoscaling.updateMode values in values.schema.json.

    Changed

    • Set the ruler’s VPA updateMode to Initial (was Recreate).
  • Observability observability-bundle v3.5.0

    Changed

    • Update alloy apps to 0.23.1 (Alloy v1.19.2).
    • Update prometheus-operator-crd to 24.0.0 (Prometheus Operator CRDs v0.94.0).
    • Update kube-prometheus-stack to 24.0.0 (chart 91.2.3, Prometheus Operator v0.94.0).

    Added

    • Point kube-prometheus-stack’s control-plane ServiceMonitors at the alloy-metrics token Secret.

    Fixed

    • KSM custom resource state: Set the Gateway API TCPRoute and UDPRoute collectors to v1, which is the version the API server serves.
  • Observability kube-prometheus-stack-app v24.0.0

    Changed

    • Upgraded chart dependency to kube-prometheus-stack-91.2.3 (from 88.6.3, via 90.0.0)
      • prometheus-operator (and prometheus-config-reloader) from v0.93.1 to v0.94.0
      • grafana subchart from 12.11.2 to 13.2.4, which switches the Grafana image from 13.2.0 to 13.2.1-distroless, enables readOnlyRootFilesystem on the Grafana container (with a new /tmp emptyDir) and sets plugins.preinstall_auto_update = false
      • k8s-sidecar (Grafana dashboard/datasource sidecar) from 2.10.1 to 2.11.2
      • kube-state-metrics subchart from 8.4.1 to 8.4.2 (KSM image unchanged at v2.20.0)
      • prometheus-node-exporter subchart from 4.56.3 to 4.57.0 (we keep nodeExporter.enabled: false, no impact)
      • Alertmanager, Prometheus, Thanos and kube-webhook-certgen images are unchanged.
    • Breaking (chart 90.0.0): the control-plane ServiceMonitors (kubelet, kube-apiserver, kube-controller-manager, kube-scheduler, kube-etcd, kube-proxy, coredns) no longer use bearerTokenFile/tlsConfig.caFile. They now authenticate through a kubernetes.io/service-account-token Secret (<release>-prometheus-token, newly created by the chart) and read the CA from the kube-root-ca.crt ConfigMap. This makes the ServiceMonitors work with arbitraryFSAccessThroughSMs.deny and with Grafana Alloy’s prometheus.operator.servicemonitors component (>= v1.19.0), which silently dropped every control-plane target before.
    • Breaking (chart 91.0.0 / operator v0.94.0): the Prometheus Operator ClusterRole no longer grants wildcard (*) verbs. Verbs are now explicit per resource group, and the operator only gets get/list/watch on the monitoring.coreos.com CRs plus writes on their /status and /finalizers subresources.
    • prometheusOperator.admissionWebhooks.matchConditions changed type from a map ({}) to a list ([]). We do not set it, so no impact.
    • Fixed the chart appVersion, which was still v0.92.0, to match the bundled Prometheus Operator (v0.94.0).
    • Added global.controlPlaneScrapeAuth to configure the Secret the control-plane ServiceMonitors authenticate with.
    • Dropped the scrape credential for coreDns, kubeEtcd and kubeProxy.
    • Removed insecureSkipVerify from the kubeControllerManager and kubeScheduler ServiceMonitors.

    Note: release the matching prometheus-operator-crd app (CRDs chart 32.0.0, Prometheus Operator v0.94.0) before this one. The CRDs shipped by this chart live in charts/crds/crds/ and are therefore only applied by Helm on install, never on upgrade.

  • Observability prometheus-operator-app v24.0.0

    Changed

    • Upgraded chart dependency to kube-prometheus-stack-91.2.3 (from 88.6.3, via 90.0.0)
      • prometheus-operator (and prometheus-config-reloader) from v0.93.1 to v0.94.0
      • grafana subchart from 12.11.2 to 13.2.4, which switches the Grafana image from 13.2.0 to 13.2.1-distroless, enables readOnlyRootFilesystem on the Grafana container (with a new /tmp emptyDir) and sets plugins.preinstall_auto_update = false
      • k8s-sidecar (Grafana dashboard/datasource sidecar) from 2.10.1 to 2.11.2
      • kube-state-metrics subchart from 8.4.1 to 8.4.2 (KSM image unchanged at v2.20.0)
      • prometheus-node-exporter subchart from 4.56.3 to 4.57.0 (we keep nodeExporter.enabled: false, no impact)
      • Alertmanager, Prometheus, Thanos and kube-webhook-certgen images are unchanged.
    • Breaking (chart 90.0.0): the control-plane ServiceMonitors (kubelet, kube-apiserver, kube-controller-manager, kube-scheduler, kube-etcd, kube-proxy, coredns) no longer use bearerTokenFile/tlsConfig.caFile. They now authenticate through a kubernetes.io/service-account-token Secret (<release>-prometheus-token, newly created by the chart) and read the CA from the kube-root-ca.crt ConfigMap. This makes the ServiceMonitors work with arbitraryFSAccessThroughSMs.deny and with Grafana Alloy’s prometheus.operator.servicemonitors component (>= v1.19.0), which silently dropped every control-plane target before.
    • Breaking (chart 91.0.0 / operator v0.94.0): the Prometheus Operator ClusterRole no longer grants wildcard (*) verbs. Verbs are now explicit per resource group, and the operator only gets get/list/watch on the monitoring.coreos.com CRs plus writes on their /status and /finalizers subresources.
    • prometheusOperator.admissionWebhooks.matchConditions changed type from a map ({}) to a list ([]). We do not set it, so no impact.
    • Fixed the chart appVersion, which was still v0.92.0, to match the bundled Prometheus Operator (v0.94.0).
    • Added global.controlPlaneScrapeAuth to configure the Secret the control-plane ServiceMonitors authenticate with.
    • Dropped the scrape credential for coreDns, kubeEtcd and kubeProxy.
    • Removed insecureSkipVerify from the kubeControllerManager and kubeScheduler ServiceMonitors.

    Note: release the matching prometheus-operator-crd app (CRDs chart 32.0.0, Prometheus Operator v0.94.0) before this one. The CRDs shipped by this chart live in charts/crds/crds/ and are therefore only applied by Helm on install, never on upgrade.

  • Observability alloy-app v0.23.1

    Added

    • Optional serviceAccountTokenSecret.enabled to render a long-lived service account token Secret.
  • Observability alloy-app v0.23.0

    Changed

    • Upgrade Alloy upstream chart from 1.10.0 to 1.12.1 (CHANGELOG)
      • This bumps the version of Alloy from 1.17.0 to 1.19.2 (CHANGELOG), which includes several CVE fixes.
      • Alloy Breaking changes
        • [prometheus.operator.servicemonitors] ServiceMonitor endpoints referencing bearerTokenFile, tlsConfig.caFile, tlsConfig.certFile or tlsConfig.keyFile are now rejected. Set allow_arbitrary_file_access = true to keep the previous behaviour.
        • [otelcol.receiver.otlp] The HTTP server now defaults to idle_timeout = "1m", read_header_timeout = "1m" and write_timeout = "30s" instead of "0s".
        • See Alloy v1.18.0 and v1.19.0 release notes
      • [beyla.ebpf] Beyla now runs as a subprocess, which raises the memory used by the Alloy container.

    Fixed

    • Allow verticalPodAutoscaler.updatePolicy to be set, including the in-place update modes.
  • Observability dashboards v4.30.0

    Added

    • Add an agent_platform sub-chart (Team Bumblebee) and its first dashboard, Giant Swarm / Agent Platform / LLM usage: tokens, estimated cost and latency for every model call that goes through the Agent Platform’s agentgateway LLM listener, broken down by agent, agent namespace and model. It reads the GenAI metrics the agentgateway data plane emits. Empty until an installation enables llmRouting in the Agent Platform values and points its agents at the listener.
    • Add a Cost accounting row to the LLM usage dashboard, for attributing spend rather than watching it: totals for the selected time range, cost per bucket stacked by agent and by model over a selectable Cost bucket period, and per-model and per-agent-and-model tables with column totals.

    Changed

    • Move the private Falco dashboard to the new Giant Swarm Security organization and rename it to Anomaly Detection, to include other data sources.

    Removed

    • Drop the Security: Falco Dashboard and Falco public dashboard boards, so no Falco dashboards remain outside the Giant Swarm Security organization.
  • Observability alloy-app v0.22.0

    Added

    • Add RBAC to allow alloy-metrics to read the alloy-vcenter-credentials secret when running vSphere or VCD clusters.
    • Add serviceWhenDisabled.enabled, keeping the Service rendered while alloy.enabled is false. Off by default.
  • Observability observability-bundle v3.4.0

    Changed

    • Values: Update Prometheus Operator CRD and Kube Prometheus Stack to v23.0.0.
  • Observability kube-prometheus-stack-app v23.0.0

    Changed

    • Chart: Upgrade kube-prometheus-stack to v88.6.3, including:
      • prometheus-operator to v0.93.1
      • kube-state-metrics to v8.4.1
      • prometheus-node-exporter to v4.56.3
      • grafana to v12.11.2