Observability

  • Changed

    • Updated Tempo dashboards to mixins v2.10
    • bugfixes in Tempo operational dashboard
    • Update Network Traffic Analysis Overview dashboard
      • Replace average network traffic gauges with total data transfer bar chart
      • Add a time period selector for data transfer periods
      • Make total visible in all panels
      • Remove datasource variable
      • Simplify traffic rate queries

    Removed

    • Remove the object-storage-operator
  • Changed

    • Upgrade grafana chart: 11.3.3 => 11.3.6
    • upgrade pg-cluster-recovery-test subchart: v0.3.0 => v0.4.0
  • Fixed

    • Fix Grafana Management Lifecycle Policy.
  • Added

    • Added new gRPC routes for Loki and Tempo write

    Changed

    • Rename mimir.writeRewritePaths → mimir.write.stripPrefixPaths to clarify that the /prometheus prefix is stripped before forwarding; add equivalent stripPrefixPaths: [] defaults to loki and tempo write config.
    • Expose Tempo gRPC backend config in values (tempo.read.grpc.backendService, tempo.read.grpc.backendPort) instead of hardcoding in the template.
    • Expose Loki and Mimir backend config in values (loki.read.backendService, loki.read.backendPort, loki.write.backendService, loki.write.backendPort, mimir.read.backendService, mimir.read.backendPort, mimir.write.backendService, mimir.write.backendPort) instead of hardcoding in the template.
    • Restructure Helm templates into per-service subdirectories (templates/loki/, templates/mimir/, templates/tempo/).
    • Share HTTPRouteFilter resources within each service: a single headers-check filter and (for Mimir/Tempo) a single rewrite filter are now referenced by all routes in that service namespace.
  • Changed

    • Upgrade grafana chart: 11.2.3 => 11.3.3
    • Upgrade grafana (appVersion): 12.4.0 => 12.4.1
  • Changed

    • Upgrade Tempo chart from to 2.4.2 to 2.6.2
      • Upgrades Tempo from 2.10.1 to 2.10.2

    Added

    • requests and limits for distributor - fixes its HPA

    Fixed

    • vulture search window reduced to 24h to avoid querying traces out of retention period
  • Fixed

    • Fix Azure storage management policy rule YAML structure for lifecycle policy.
  • Added

    • Add httpRouteFilter support to Gateway API routes, enabling HTTPRouteFilter resources for tenant header enforcement (X-Scope-OrgID).
    • Add backendTrafficPolicy support to Gateway API routes. Uses a free-form spec object to allow any Envoy Gateway BackendTrafficPolicy fields (timeout, requestBuffer, circuitBreaker, etc.) rather than hardcoding a single field.
  • Added

    • Pre-configure httpRouteFilter.directResponse default (400 Bad Request) so that enabling httpRouteFilter returns a meaningful response without requiring callers to specify the response body.
  • Changed

    • Enable priority class by default to ensure fluentbit can preempt lower priority pods on overloaded nodes.