1. Docs
  2. Changes and releases
  3. AI Agents

AI Agents

  • AI Agents agent-platform v4.18.0

    Added

    • The Substrate WorkerPool’s CPU generation pin — karpenter.k8s.aws/instance-generation next to the vendor in kagent.substrateWorkerPool.template.nodeSelector — is documented, render-guarded and verified (giantswarm/agent-platform#457, the caveat #430 recorded). One pool runs one CPU feature set: an actor’s golden snapshot is a gVisor checkpoint that restores only on a host whose CPU offers every feature it recorded, and #429’s vendor pin left the generation open — on gazelle (2026-09-14) Karpenter put two workers on an m7a.2xlarge (generation 7, AVX-512) and two on c5ad.xlarge (generation 5), the goldens were taken on the m7a workers, and every cold turn placed on a c5ad worker failed incompatible FeatureSet: missing features: … avx512f … twice (30 s each), then SendStreamingMessage Internal, “the turn failed” in Slack; the next morning’s pool (c6a, c5a ×2, m6a) restored everywhere only because the goldens had been re-taken on a gen-5 worker. The pin is a second label in the same map, set per installation (the fleet template renders it from the installation’s values the way it renders the vendor): pick a generation that is one CPU model across its families — for AMD on AWS 6 (c6a/m6a/r6a, EPYC Milan) or 7 (c7a/m7a/r7a, Genoa); 5 mixes Naples (m5a/r5a) with Rome (c5a/c5ad) and is no feature-set pin — and one the goldens restore on (an older generation’s golden restores on a newer one, never the reverse), and check the spot pool it leaves (the generation’s families × the NodePool’s sizes; three families is the floor). The meta chart now refuses a nodeSelector value that is not a string naming the key (agent-platform.validateWorkerPool): the label is "6", and a bare 6 rendered, passed the open kagent schema and would have failed only when helm-controller applied the kagent release on every installation carrying it. make verify-workerpool (in CI) asserts the default forwards the architecture alone, an installation’s three-label pin reaches the kagent release verbatim with every value a string, the unquoted generation fails the render, and the kagent chart the range resolves to renders the pin unchanged into the one WorkerPool’s spec.template.nodeSelector — the lab has no Karpenter labels, so this render is where the pin is proven. Docs: values.yaml, the chart README, the root README’s Substrate section and UPGRADE.md name the generation next to the vendor. A restore that fails with incompatible FeatureSet is retried on the same worker; placing it on another worker is Substrate’s (giantswarm/giantswarm#37742 row 12), not this chart’s. No rendered object changes for an installation that sets no generation.
  • AI Agents muster v5.19.14

    Fixed

    • The core catalogue is refreshed in the background, not on the call that finds it old (#1231). The aggregator’s own tools (core_*, workflow_<name>) are kept for five minutes as the backstop for a definition change no invalidation saw; the meta-tool call that found them older paid the rebuild – one list of every Workflow resource from the API server plus their conversion, 1.9-2.05 s with 282 workflows – while a warm session had been answering the same calls in 7-20 ms. Now that call is served the catalogue it has and starts one rebuild in its own goroutine; concurrent calls share it, and the rebuilt catalogue is swapped in when it lands. A call without a catalogue – the first of the process, or after a workflow was created, updated or deleted – still waits for the rebuild, so a definition change stays visible on the very next call; a change that arrives while a rebuild runs makes it build again before anything is served. A change in the server registry – a backend server registered, lost, or re-registered by the reconciler’s status sync – no longer invalidates the core catalogue either: it depends on the tool providers alone, not on which servers are registered, yet every registry change ran the capability refresh that threw it away, so on an installation with 87 servers the meta-tool calls after a muster restart paid the rebuild again and again (1.2-3.3 s calls minutes apart, none of them at an age boundary). Every rebuild is logged at debug (Core catalogue rebuilt, with its trigger and duration_s). MUSTER_CORE_CATALOGUE_MAX_AGE overrides the age (the integration test harness runs it at 3 s).
  • AI Agents muster v5.19.13

    Fixed

    • A muster whose configured Valkey is unreachable at startup no longer serves sessions on in-memory stores for the life of the pod (#1229). The session auth and capability stores were built once at startup; when the Valkey configured in oauth.server.storage did not answer at that moment – muster and Valkey starting together, a Valkey OOM or a node roll while muster restarts – muster logged Failed to create Valkey client for session stores, falling back to in-memory and kept in-memory stores: muster:cap:* and muster:auth:* never appeared in Valkey while sessions worked, the next restart lost every session’s view and SSO state, and the OAuth token store, which retries its own connection, ended up in Valkey regardless. Now muster dials the configured Valkey again with backoff (1 s doubling to 8 s) for up to 20 s (under the chart’s 30 s liveness window), logging Valkey for the session stores did not answer, retrying in between and serving nothing until it answers; when it stays away the start fails with Configured Valkey did not answer; refusing to serve sessions on in-memory stores and muster serve exits non-zero, so the kubelet restarts the pod. In-memory stores remain the behaviour of a deployment that configured no Valkey. The backend in use is published as the gauge muster_session_store_backend{backend="valkey|memory"} (1 for the backend in use, 0 for the other). The Valkey client’s client-side cache, which no store reads through, is off (CLIENT TRACKING is no longer requested per connection).
  • AI Agents agent-platform v4.17.0

    Changed

    • agent-platform-connectivity follows the meta chart’s own version; a pin on it is refused (giantswarm/agent-platform#450, the root cause behind #431, #441 and #339). The wiring chart is published off the same git tag as this chart — two charts, one commit, one version — yet its versionRange was a range of its own (>=4.0.0 <5.0.0) that a BOM pinned separately, so every BOM’s connectivity lagged the moment the meta chart moved: three releases in a row forwarded a key (gateway.parameters.podAnnotations, vm-manager / vmManager, postgres.affinity / imagePullSecrets) that the pinned connectivity’s closed schema refused, and the release failed on every BOM-pinned installation. The example BOM’s own pin was fourteen releases behind. The connectivity entry is now releasedWithChart: true with an empty versionRange: the meta chart renders the OCIRepository at its own exact version (agent-platform.chartVersion — build metadata dropped, a pre-release kept, the same derivation the self range floors on), so Flux rolls the connectivity release exactly when the meta chart rolls and never ahead or behind. A versionRange on such an entry is refused when it would pull from the repository this chart’s releases come from (gitops.self.repository) with no semverFilter — that is the lagging pin; the two development shapes keep their knobs (a semverFilter for a dev channel, another repository for a chart pushed by hand — the ATS’s in-cluster registry, a lab registry). examples/customer-bom.yaml no longer pins connectivity and tests/verify-components.py refuses one; the ATS smoke and own-flux scenarios point the component at the in-cluster registry and let the version derive, proving it against a real source-controller with a pre-release version. Every installation sees its connectivity OCIRepository move from the range to the exact version of the meta chart it runs — the same chart the range resolved to for an installation on the current release, an upgrade for one whose connectivity lagged (UPGRADE.md).
    • make verify-components-charts renders every component of the meta chart’s roster, and the BOM has to pin all of it (giantswarm/agent-platform#278 follow-up, #450). The check rendered thirteen components named in the script — the seven extras, the two managers, the kagent and Substrate lines — so agent-platform-connectivity, klaus-gateway, muster, agentgateway, valkey, agent-platform-mcps, agent-sandbox, dicebear and vm-manager were never rendered against the values the meta chart forwards them; that list is exactly what let #278’s three breaks reach main. The set is now the meta chart’s own roster (every components.* entry of values.yaml with a chart), and examples/customer-bom.yaml must match it both ways: a roster entry the BOM does not pin, a pin that names no component, a pin the render carries differently than the BOM’s line, a pin no registry publishes, and a pin on a chart released with the meta chart each FAIL. Every component is rendered at the range’s resolution with the defaults’ forwarded values and at the BOM pin with the BOM render’s forwarded values (one render when both agree); agent-platform-connectivity, released with the chart, is rendered from the working tree — the pair an installation gets is always the pair in the commit, and a key added to both charts at once exists nowhere else until the tag is cut (#339). The BOM and the roster are read as YAML (PyYAML is in the CI image), every Helm call is bounded, and the four UNRELEASED entries whose releases exist are gone: a range that admits nothing published is still rendered against the newest chart the line has while UNRELEASED names the release it waits for, a BOM pin never is. The inputs an installation supplies and the meta chart leaves empty (the resource servers’ OAuth clients, a public Gateway for the routes, valkey’s ACL secret) are ALL_ON_INPUTS, deliberately apart from QUICKSTART, which tests/verify-kagent-tools-namespace.py imports for renders with a much smaller component set. 22 components, 43 renders.

    Removed

    • tests/verify-bom-charts.py and its make verify-meta step (giantswarm/agent-platform#450; added by #278, carried #339’s working-tree rule for the connectivity chart). make verify-components-charts does strictly more: it renders the charts through where that one only proved the schema accepted the values (reporting the rest as needs-input), it covers the range’s resolution as well as the pin, and it renders the pin with the BOM render’s forwarded values. The BOM-completeness guard, the pin-reaches-the-render assertion and the working-tree rule moved with it.
  • AI Agents muster v5.19.12

    Fixed

    Full Changelog: https://github.com/giantswarm/muster/compare/v5.19.11...v5.19.12

  • AI Agents muster v5.19.11

    Fixed

    • Meta-tool calls no longer rebuild the session’s catalogue from the definition source and the capability store on every call (#1225). call_tool, describe_tool, filter_tools, list_tools and the resource/prompt meta-tools each listed the caller’s catalogue afresh: every workflow definition was fetched from the API server twice per call (once for the tool’s arguments, once to derive its read-only hint – 8182 GET workflows in the two minutes of one agent turn with 282 workflows), and every session-authenticated server’s capabilities were read from Valkey with their own HGET and GET (~160 round trips per call with 81 such servers). Together that was a floor of about 3 s per meta-tool call on a warm session while the backend answered in 22 ms. Now the aggregator’s own tools (core_*, workflow_<name>) are built once and kept until a workflow definition changes – through muster’s tools or, for a resource applied outside them, through the workflow reconciler – with the step tools carried on the tool so the read-only hint is derived in-process; a session’s capabilities are read with one HGETALL, and the content-addressed documents behind the references are decoded once per process and shared by every session that references them; the servers a session is authenticated to are read with one HKEYS. A warm meta-tool call costs two Valkey commands regardless of how many servers are registered. Toolset scoping, session-scoped visibility and the capability freshness mechanisms are unchanged.
    • tools/call response and Error log lines carry duration_s, the time since the request reached the transport, so a slow call is visible in the log without subtracting the request line’s timestamp (#1225).
  • AI Agents agent-platform v4.16.0

    Changed

    • Anthropic prompt caching is on for every Anthropic ModelConfig the platform renders; the kagent line moves to v0.11.0-gs.15 (giantswarm/giantswarm#37788). The kagent Go ADK sent every Anthropic request without cache_control, so each model call of an agent loop re-processed its whole prefix — system prompt, tool schemas, the growing history — at full input price, with a time to first token that followed the uncached prefix (gazelle, 2026-09-14: 0.7-1.5 s at 6 k input tokens, 9.7-13.5 s at 33-40 k; one twelve-call status question re-sent 340 k input tokens for $0.73). kagent 0.11.0-gs.15 carries kagent-dev/kagent#2788: spec.anthropic.promptCaching / cacheTTL on the ModelConfig CRD, three cache_control breakpoints in the Go ADK (the last tool definition, the last system block, the last content block of the latest turn), cache_read_input_tokens in the usage. The meta chart sets kagent.providers.anthropic.config.promptCaching: true and cacheTTL: "5m", which the kagent chart renders into the default ModelConfig next to the LLM cut-over’s baseUrl; the connectivity chart’s kagent.modelConfigs[] Anthropic entries inherit both from that block (an entry’s own promptCaching / cacheTTL win, false included; a Bedrock entry takes its own keys under spec.bedrock; any other provider refuses the keys naming the entry, and a cacheTTL outside 5m / 1h fails the render — the API server would prune or refuse them after the render said nothing). components.kagent / components.kagent-crds move to >=0.11.0-gs.15 <0.11.1-0: the floor, because an older CRD prunes the two fields at admission and the models stay uncached in silence. make verify-prompt-caching asserts the forwarding to both releases, the inheritance, the precedence, the Bedrock block, the chart-alone default and the guards; tests/verify-kagent-crds.py validates the cached shape against the gs.15 CRD; the BOM and tests/verify-components.py follow the pin. Lab proof (agentlab on 4.15.5 with the LLM listener on, the kagent controller and Go ADK built from the cherry-pick, promptCaching: true on the default ModelConfig, claude-sonnet-4-6, a five-call tool turn on a ~4.3 k-token prefix): stock gs.14 wrote and read nothing (five calls, $0.085); on the patch the first call wrote 4 302 tokens and every later call read 97-99 % of its prompt from the cache (the turn $0.037), and the next turn’s first call read 3 961 tokens the previous turn had written ($0.019 for that turn); agentgateway’s LLM listener passes cache_control through unchanged. Every installation rolls kagent once (the controller, then the platform Harness recompiles every admitted AgentTemplate) and its Anthropic ModelConfigs gain the two fields, which recompiles their templates again; from then on the first call of a turn writes the cache (1.25×) and every call after reads it (0.1×) — UPGRADE.md, including the opt-out.
  • AI Agents agent-platform v4.15.6

    Fixed

    • Every agent turn ended 3 s late: the actors could not reach the OTLP gateway, and the Go ADK’s pre-response trace flush spent its full deadline on every turn (giantswarm/agent-platform#456; gazelle, 2026-09-14, nine of nine turns). The kagent controller sets KAGENT_PRE_RESPONSE_TRACE_FLUSH=true on Substrate actors (a checkpoint runtime has no later export window), so the Go ADK force-flushes a turn’s spans before the A2A response closes, with a detached deadline of KAGENT_TRACE_FLUSH_TIMEOUT_MS (3 s by default). The actors’ exporters point at the OTLP gateway kagent.otel names (http://otlp-gateway.kube-system.svc:4317), but the connectivity chart’s substrate-atenet-egress policy — the actors’ egress allow-list, since every connection an actor opens leaves through Substrate’s egress gateway — opened nothing towards it: every flush ran into traces export: context deadline exceeded at exactly +3.000 s, then the checkpoint, then the controller’s SendStreamingMessage closed; the caller (Swarmgeist’s ✅, the portal’s receipt) saw a one-word answer “done” 3.3–3.6 s after the text. The actors’ log exporter was broken twice over: the controller forwards only its tracing variables to the actors, so the Go ADK’s built-in log exporter (HTTP, /v1/logs) posted to the gRPC port and every batch ended in EOF. Three changes, one path: (1) the connectivity chart opens the OTLP gateway on the egress gateway and — through the same helper, agent-platform.kagent.otlpEgress — on the kagent controller’s egress policy, one rule per distinct destination of the kagent.otel signals that are on, selecting the pods of the endpoint’s namespace (<svc>.<ns>.svc[.cluster.local]) on the endpoint’s port (kube-system on 4317 by default, once for both signals; a second endpoint a second rule; an endpoint that is not a Service address the cluster entity on its port, as the controller’s rule was for every endpoint before); nothing when both signals resolve off. (2) The platform Harness’s env (kagent.harness.env) gains OTEL_LOGGING_ENABLED=true — the Go ADK exports the actors’ logs over gRPC to the tracing endpoint instead of the HTTP fallback — and OTEL_EXPORTER_OTLP_HEADERS=X-Scope-OrgID=giantswarm, the tenant header the controller sends and deliberately keeps from the actors; the meta chart drops the former when kagent.otel.logging resolves off and the latter when both exporters do (agent-platform.shape.dropEnv, the rule the controller’s header already followed). (3) KAGENT_TRACE_FLUSH_TIMEOUT_MS=500 on the Harness caps the flush: an in-cluster export takes tens of milliseconds, and a slow or unreachable collector can now cost a turn half a second, not three. make verify-actor-telemetry-egress asserts the rules (both policies, the default, a second endpoint, the port following the protocol, both signals off, auto off without the monitoring API, one signal on, the kubernetes flavour rendering no Substrate egress policy); tests/verify-kagent-harness.py the env in every exporter state. The lab renders no network policies (kind runs no Cilium): the policy is render-verified and live-verified on gazelle (hubble observe --verdict DROPPED on the egress gateway stays empty during a turn); the flush cap is lab-verified against an unreachable collector. Substrate’s own telemetry (the ateom worker’s periodic failed to upload metrics: … dial tcp [::1]:4317, substrate.otel.endpoint unset in the meta chart, the atelet OTLP relay socket absent on a worker that started before its atelet) is a separate finding, not on a turn’s path — a follow-up issue.
  • AI Agents agent-platform v4.15.5

    Changed

    • The agentgateway data plane moves to the line’s v1.5.1-gs.4 (agentgateway.proxy.image.tag in the connectivity chart, rendered into AgentgatewayParameters; giantswarm/giantswarm#37758): the Giant Swarm line of agentgateway re-pinned from upstream v1.5.0 onto upstream main @ c1d24607 (2026-09-14 — the substrate protocol of upstream #3237/#3318/#3409/#3428 that the Substrate line’s v0.0.29 router speaks; upstream has released nothing since v1.5.0), still carrying the GRPCRoute method-match translation and the resuming-actor admission. The giantswarm/agentgateway packaging chart moved the controller to the same release (giantswarm/agentgateway#52, released 2.1.2), so the data plane follows it here — controller and data plane are one build again. Proven in agentlab with both on gs.4 under 4.15.2: platform-test 7/7, a2a-test 5/5 (the GRPCRoute and its JWT policy through the edge), toolsets-test 11/11. Every installation with the agentgateway component on rolls its data-plane pods once. The Substrate and kagent line re-pins that belong to the same move follow in the next release.
    • The three Bumblebee upstream lines re-pin together — agentgateway v1.5.1-gs.4, Substrate v0.0.30-gs.1, kagent v0.11.0-gs.14 (giantswarm/giantswarm#37742, #37010, #37757, #37758): components.substrate / components.substrate-crds move to >=0.0.30-gs.1 <0.0.31-0 (the Substrate line on upstream kagent-dev/substrate v0.0.29, its ten carried patches replayed — the egress gateway’s frontend policy is upstream’s own from v0.0.29 under agentgateway#3318’s name, kagent-dev/substrate#33 fell away — and its router and egress gateway on the agentgateway line’s v1.5.1-gs.4), components.kagent / components.kagent-crds to >=0.11.0-gs.14 <0.11.1-0 (the kagent line on upstream kagent-dev/kagent main @ 800015de, 2026-09-14 — kagent-dev/kagent#2802 moves kagent to Substrate v0.0.29 and its a2a gateway addresses an actor by the ate-target-actor header, which the 0.0.27 router did not know, so a kagent past #2802 on the previous Substrate fails every turn; the #2739 cherry-pick fell away, upstream has it — whose chart stamps the 0.0.30-gs.1 worker image) (the agentgateway line’s v1.5.1-gs.4 — upstream main @ c1d24607 — is the release the controller and data plane already run since the previous entry and the Substrate chart pins for its router). Proven in agentlab on the three lines’ dev builds together on this chart’s 4.15.2 (an upgrade from 0.0.27-gs.9 / 0.11.0-gs.13 / v1.5.1-gs.3): platform-test, agents-test, skills-test, toolsets-test, a2a-test and a headless turn through the edge. Every installation on the 4.x line rolls once: the Substrate control plane and the WorkerPool (the workers restart — an actor mid-turn on a rolled worker loses that turn, every golden snapshot is retaken on the new worker image), the kagent controller and UI — UPGRADE.md. BOM, tests/verify-components.py and tests/verify-kagent-crds.py follow the pins.
  • AI Agents agent-platform v4.15.4

    Changed

    • (deps) Update gsoci.azurecr.io/giantswarm/agent-manager docker tag to v1.1.4 in #459 by @renovate[bot]

    Full Changelog: https://github.com/giantswarm/agent-platform/compare/v4.15.3...v4.15.4