AI Agents

  • Fixed

    • The s3proxy façade declares its ephemeral storage and bounds its emptyDirs (giantswarm/agent-platform#438; glean on connectivity 4.10.3 and 4.10.4, 2026-09-13). The substrate-s3proxy container mounts two emptyDirs (/tmp for the JVM, the image’s /data) and carried no ephemeral-storage requests or limits, so Kyverno’s require-emptydir-requests-and-limits (Audit on the fleet) reported Containers mounting emptyDir volumes must specify requests and limits for ephemeral-storage on every rollout — the one Kyverno finding the platform left on a fresh 4.x installation, and a refusal on a cluster that enforces the policy. kagent.harness.snapshotStore.s3proxy.resources now defaults requests.ephemeral-storage: 256Mi and limits.ephemeral-storage: 1Gi in both charts (the meta chart forwards the block; the façade streams, nothing of a snapshot lands on its disk), both emptyDirs take the limit as their sizeLimit, and the render refuses a resources block without either field, naming the key. make verify-substrate-store asserts the fields and the bound on the capz and the façade-alone shapes, the knob, the forwarded default and the guard; tests/verify-components.py holds the two charts’ resources defaults together. An installation with the façade rolls its two substrate-s3proxy pods once (a pod template change; the rolling update surges the new pod first) — UPGRADE.md. Lab proof (agentlab, the façade alone against Azurite, hack/agentlab/): a lab on connectivity 4.10.4 upgraded to this branch’s dev build rolled the façade in 6 s (new ReplicaSet, 2/2 Ready, old at 0) with ephemeral-storage 256Mi/1Gi and both emptyDirs sizeLimit: 1Gi on the live pods; an AgentTemplate created right after went Ready on the platform Harness in 5 s with its golden snapshot (5 objects, 1.3 MiB) written through the rolled façade into Azurite; agentlab platform-test all PASS before and after. The Kyverno half is fleet-only (agentlab runs no Kyverno): the acceptance is no require-emptydir-requests-and-limits event on glean once the release rolls.
  • Fixed

    Full Changelog: https://github.com/giantswarm/muster/compare/v5.19.3...v5.19.4

  • Fixed

    Full Changelog: https://github.com/giantswarm/klaus-operator/compare/v0.0.159...v0.0.160

  • Fixed

    • A suspended remote MCPServer is stopped once, not on every resync tick (#1212). The stop of a remote server (streamable-http, sse) settles in Disconnected, never in Stopped, but the reconciler’s suspend path only recognised Stopped and Stopping as done – so a server with spec.suspended: true was stopped again every 30 s for as long as it stayed suspended: Suspending MCPServer service <name> (spec.suspended=true) and Stopped service: <name> in the log, an MCPServerStopped Kubernetes event and a second reconcile from the state change on each tick, 13 days of it for one server on a management cluster. reconcileSuspend now returns early for Disconnected as well; Failed, Error and Unreachable are still stopped, so suspending a server whose endpoint is down still ends its reconnect schedule. Service.Stop has the same guard: called on a remote server that is already Disconnected it returns without a state write and without an event, as it always did for a local server that is Stopped. Resume (spec.suspended: false) is unchanged. Scenario: mcpserver-suspend-remote-stops-once; muster test runs instances with MUSTER_RECONCILER_RESYNC_INTERVAL=2s (a new Go-duration override of the reconciler’s 30 s resync) and instance_logs gained occurrences, an exact line count per substring.
  • Changed

    • An in-cluster jwtAuthentication.jwks.host in a namespace, or on a port, that gateway.jwksEgress does not name now fails the render, under networkPolicy.enabled. That key renders one rule, for one namespace on one port, so a host elsewhere reached nothing while the render stayed green and every request read 401 token uses the unknown key. The message names the value to set. Both guards stand down for a host reached by address: gateway.jwksEgress.external.cidrs non-empty on an external.port equal to the route’s jwks.port, which is how a second in-cluster issuer is carried — the one rule covers one namespace on one port, so two routes on in-cluster hosts in different namespaces cannot both be satisfied. gateway.jwksEgress.podSelector is read by no guard, because a hostname carries no pod labels. An installation whose issuer is dex.giantswarm.svc.cluster.local on 5556, the default, is unaffected.
  • Fixed

    • A deactivated MCPServer refuses sign-ins and reports itself down (#1211). An OAuth MCPServer with spec.suspended: true (what core_service_stop and the portal’s Deactivate write) keeps its pending-auth registry entry so that sign-ins work again once it is activated – but that entry also let core_auth_login hand out a sign-in link while the server was suspended. The person completed the flow, the session connected, the event handler failed a global registration (Failed to register MCP server … no MCP client available (service state inconsistent)), the reconciler stopped the service again within milliseconds, and auth://status then said connected with a tools count while the session’s tool list had nothing for the server; a session that had signed in before the deactivation saw the same contradiction. Now core_auth_login refuses a suspended server with Server '<name>' is deactivated (spec.suspended=true); activate it with core_service_start before signing in. and creates no challenge; the OAuth callback for a server deactivated between challenge and callback stores the token and establishes no connection (the first core_auth_login after core_service_start connects with it, no browser needed); auth://status reports a server whose service is down as disconnected whatever the session’s auth mark says, with a new "suspended": true flag naming the reason (muster auth status prints Deactivated); and list_tools no longer names a down server under servers_requiring_auth. The spec is read through the MCPServer manager – the source the reconciler acts on – so the aggregator keeps no copy of it. The service state inconsistent ERROR turned out not to be the suspension’s doing: the event handler tried a global registration on every per-session OAuth sign-in (the skip covered SSO servers only), failed because such a service holds no client, and emitted a MCPServerToolsUnavailable event the moment a person connected; it now skips a server that authenticates per session the way it skips SSO servers. Scenario: mcpserver-suspended-refuses-auth-login; test_simulate_oauth_callback gains an auth_url argument to complete a challenge an earlier step obtained.
  • Fixed

    • Duration histograms use bucket boundaries for seconds (#995). muster_tool_call_duration_seconds, muster_downstream_tool_call_duration_seconds and muster_workflow_execution_duration_seconds are recorded in seconds but inherited the OTel SDK’s default boundaries (0 … 10000), which are spaced for milliseconds: every observation landed in the first bucket and the other 14 series stayed empty, so histogram_quantile over them could resolve nothing finer than “under 5 seconds”. A sdkmetric.View matched on unit s now applies 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120, 300 to every seconds-unit histogram, current and future. The boundaries are documented in docs/explanation/observability.md next to the p95 example query.
  • Changed

    • describe_tool states that the tool it describes is only reachable through call_tool. Its response gained an invocation line — Call it through the call_tool meta-tool: call_tool with {"name": "<tool>", "arguments": {...}}, arguments per inputSchema. Tools inside muster are not callable by name directly — only the meta-tools are. — which the meta-tool’s own description names, next to what it returns, and call_tool’s own description now states the rule itself (“Aggregated tools are callable only this way, never by their own name”), so a model reads it when it decides how to execute rather than only after a lookup. An MCP client sees only the meta-tools, so a model that took a name out of describe_tool and issued it as a tool call of its own got an error with nothing in the detail it had just read to recover from. The field is additive: description, inputSchema, server, kind and annotations are unchanged, and the consumers that parse the response (the muster CLI, the REPL describe) ignore what they do not know. Scenario: list-tools-paged.
  • Fixed

    • The s3proxy façade authenticates to Azure Blob with Workload Identity (giantswarm/agent-platform#436; found on glean’s cut-over, 2026-09-13). With kagent.harness.snapshotStore.crossplane.provider: capz the connectivity chart left JCLOUDS_CREDENTIAL unset so that s3proxy would defer to DefaultAzureCredential — but the image (gaul/s3proxy 4.1.1) carries JCLOUDS_CREDENTIAL="remote-credential" as a Dockerfile default, and s3proxy takes the DefaultAzureCredential path only while the credential is empty: the façade signed every request as the account with that literal as the shared key and Azure’s SDK refused it on the base64 decode ('base64Key' was not a valid Base64 scheme … Illegal base64 character 2d on every PUT …/snapshots/…), so every golden boot crashed before its snapshot was taken. The capz branch now sets JCLOUDS_CREDENTIAL: "" explicitly next to AZURE_CLIENT_ID / AZURE_TENANT_ID; the account-key branch is unchanged. make verify-substrate-store asserts both shapes. Live proof on glean: with the variable empty the golden snapshot of sre-agent landed in giantswarm-glean-substrate through the façade on the next boot (Workload Identity end to end, the half agentlab cannot exercise — its façade proof runs the account-key branch against Azurite).
  • Changed

    • The Substrate line re-pins to v0.0.27-gs.9, the kagent line to v0.11.0-gs.12 (giantswarm/giantswarm#37773): components.substrate / components.substrate-crds move to >=0.0.27-gs.9 <0.0.28-0, components.kagent / components.kagent-crds to >=0.11.0-gs.12 <0.11.1-0 — the kagent release whose chart stamps the 0.0.27-gs.9 worker image, where the gVisor worker’s terminate collects a workload whose runsc containers are already gone (giantswarm/substrate#30). Before it, an AgentTemplate changed while its previous golden boot was crash-looping left the superseded revision uncollectable — failed to collect runtime revision … runsc state: exit status 128 every minute per template — and its golden actor pinned one worker of the pool for good (gazelle 2026-09-13: two of four). Proven in agentlab on the fork’s dev build: on 0.0.27-gs.7 the superseded revision was never collected in 56 minutes; on the patch the sweep after the template change collected it and freed the worker. The WorkerPool rolls once with the kagent release, which by itself frees the workers an installation on 4.10.x has pinned. UPGRADE.md carries the recognition and the by-hand recipe (delete the hosting worker pod) for a Substrate release before gs.9. BOM, tests/verify-components.py and tests/verify-kagent-crds.py follow the pins.