AI Agents
Added
components.gpu-operator: NVIDIA’s GPU operator as a chart component, off by default and independent of model serving (giantswarm/agent-platform#327; bumblebee-plans#46, the plan’s D3; epic giantswarm/giantswarm#37639). One OCIRepository + HelmRelease of Giant Swarm’s gpu-operator-app — the catalog’s gpu-operator wrapper chart on 1.x — into kube-system (the release history there too, so the GPU guide’s hand-installed release upgrades in place) with crds: CreateReplace, no dependsOn, gitops.target.kubeConfig.secretRef honoured. The operator’s configuration is the new gpu-operator: block, forwarded under the wrapper’s subchart key — a table of two rows and no third: Flatcar, driver and toolkit off (the default); nodes with a pre-installed driver, gpu-operator.toolkit.enabled: true. In every configuration the operator creates the nvidia RuntimeClass GPU pods use (modelServing.serving.runtimeClassName: nvidia where the platform’s own release serves), so the chart adds none. One owner per cluster: with the component on, a ClusterPolicy, HelmRelease or App of the operator that is not this release’s fails the render naming what was seen and the handover — delete cluster-manager’s <cluster>-gpu-operator (its detection then sees the component and never re-creates it) or the hand-installed release, then switch the toggle on, or leave it off; the ClusterPolicy CRD is under ownedCrds as the backstop. The default render is unchanged but for the roster entry forwarded to connectivity (components.gpu-operator.enabled: false), which make verify-target’s golden now drops as the one difference a new component is allowed. make verify-gpu-operator asserts the offline half; tests/fixtures/gpu-operator-foreign-owner.yaml shows the refusal on a cluster; examples/customer-bom.yaml pins 1.3.0. README “The GPU operator”.
Added
- One release of this chart per target cluster: the target kubeconfig knob,
enabled for muster and dicebear, one owner per cluster-scoped component, the slice fixtures (giantswarm/agent-platform#328; bumblebee-plans#46, epic giantswarm/giantswarm#37639). The serving slice (#326) and the runtime slice (#317) are values profiles cluster-manager installs as one <cluster>-agent-platform release per target cluster, the slices as toggles in its values — beside the platform’s own release on the installation’s cluster, or onto a workload cluster that runs no Flux. Three things in the chart stood in the way. New: gitops.target.kubeConfig.secretRef (name, key) — when set, every component HelmRelease the loop renders carries spec.kubeConfig.secretRef verbatim (the connectivity release included), the field shape the fleet installs every workload-cluster app with, so the installation’s helm-controller installs each component into the target from the OCIRepositories in the release namespace, dependsOn intact, the connectivity chart’s lookup guards evaluated against the target; unset, the render is byte-identical to before; refused with the bundled engine on (agent-platform.validateTarget — no Flux is installed into a workload cluster or into a cluster that runs one), and no Helm hook renders with it (the kagent storage-version pair would run against the installation’s CRDs). components.muster.enabled and components.dicebear.enabled (default true; the two were force-on) — off renders no release of theirs and the roster tells the connectivity chart, which now drops every reference: muster’s /mcp HTTPRoute, the -mcp-egress policy, the agentgateway data-plane and controller rules, the kagent controller and Substrate rules that select muster’s pods (the managers’ already read the roster); the avatars host in the portal’s CSP. components.<name>.ownedCrds — a CRD the component installs that already exists and whose Flux labels (helm.toolkit.fluxcd.io/name, /namespace, which helm-controller stamps on every object of a release, crds/ included) name another HelmRelease fails the render naming the release to remove first (agent-platform.validateCrdOwners; agentgateway, kserve-crd, kserve-llmisvc-crd, kagent-crds and muster carry one representative CRD each; skipped with the target knob, where the lookups see the installation; a CRD without the labels is left to Helm as before). make verify-target (tests/verify-target.py) asserts the offline half: the stamp and the byte-identity of five renders against origin/main, the toggles on both charts, the engine guard, the hooks, and the serving- and runtime-shaped toggle sets (ci/test-slice-serving-values.yaml, ci/test-slice-runtime-values.yaml, ci/test-target-values.yaml — the profiles themselves are #326’s and #317’s) alone, combined (the union; the first slice’s documents unchanged when the second is switched on — an in-place upgrade) and targeted, agentgateway off beside the platform’s release and on for a workload cluster; the two lookup guards are asserted on a cluster (README, “One release per target cluster”). Default render unchanged.
Added
muster test: an installation-shaped scale fixture with budgets enforced in CI. internal/testing/fixtures/scale is a committed, generated fixture – 87 MCPServers (84 session-authenticated, in five families with an instanceArg on 28 installations, three in-house), 18 distinct capability documents shared by the family members, 282 workflows over their tools, 450 session records; every name invented, nothing from an installation – rendered by go generate from a seeded generator, with a test that fails when the committed YAML drifts. A scenario boots from it with pre_configuration.fixture: scale and adds or overlays on top. Budgets are hard limits with headroom over the value measured on CI’s medium executor, never “faster than last run”, and a failing budget names the metric, the measured value and the limit. In Go (internal/aggregator, run by make test): a warm list_tools, filter_tools, describe_tool or call_tool over the fixture costs at most 100 ms (measured 1–2 ms), 2 store commands (HGETALL, HKEYS) and 0 definition reads (API-server requests) – the catalogue is built once; list_tools’ default page stays under 40 KB (measured 14 KB); the capability store holds at most 16 KB per session over 450 sessions (measured 7.2 KB: 84 references and a share of 18 shared documents). On the wire (scenario scale-fixture-budgets, on a Valkey stand-in): the default page under 40 KB, under 32 KB of capability store per session (measured 9.9 KB over 8 sessions), a warm meta-tool request costing the store a constant 16 commands at most (measured 11: the two of the catalogue plus the OAuth middleware’s per-request session bookkeeping, nothing per server), and a new session’s first request answered within 2 s while its fan-out connects 84 servers two of which take four seconds. New test tools test_measure_meta_tool (duration, response bytes, store commands by name per call) and test_valkey_footprint (the store’s content by prefix, capability bytes per session); a new expectation json_path_max bounds numeric response fields, and a step whose expectations are not met now reports which one and how (json_path_max "valkey_commands": measured 177, budget 16) instead of “expectations not met”. The scenario fails on the release that had each bug, at the budget’s step: list_tools unbounded on v5.15.6 (response_bytes 61,651 against 40,960, #1193); every session storing every server’s document inline on v5.19.4 (capability_bytes_per_session 105,332 against 32,768, 672 inline entries and no shared document, #1217); a listing reading every server’s entry and document on its own on v5.19.10 (valkey_commands 177 against 16: 84 HGET and 87 GET per call, #1225); a new session’s first request held by the fan-out on v5.19.14 (4.02 s against a 2 s max_duration, #1226). Documented in “Installation scale: the fixture and the budgets” of docs/contributing/testing/scenarios.md. (#1240)
Fixed
- A
forEach can iterate a step result’s field. items: "{{ .results.<id>.<field> }}" failed every workflow with items expression … resolved to string, expected a list: a reference deeper than one key below .input, .results or .vars was rendered as text, so only a list passed in as a workflow argument could be iterated. Template references that are a pure path now keep their Go type at any depth – the navigator the spec.output template already uses – so a forEach over the pods a previous step listed works, {{ .vars.pod.name }} inside the loop body is the string it names, and a numeric field referenced as a tool argument arrives as a number. Anything more than a pure path still renders to text. The returned document also showed the loop’s last result on every iteration’s record (the records share the body step’s ID); each record now carries its own iteration’s result and an iteration index. Scenario workflow-foreach-step-result-items.
Added
muster self-update --check reports the running and the latest release without installing
anything and exits with status 125 when a newer one exists (devctl’s convention for
version check), for scripts. Every command a person runs starts with a one-line hint on stderr
while a newer release is out – a hint, never a gate: the GitHub round trip is capped at two
seconds, its answer is cached for an hour under the user cache directory
(~/.cache/muster/latest-release.json on Linux, ~/Library/Caches/muster/ on macOS) and a failed
attempt is remembered for ten minutes, so a machine without internet is not held up.
MUSTER_NO_UPDATE_CHECK=1 silences the hint; serve, standalone, agent, test, version,
self-update, help and completion never print it, and dev builds never check. The same
internal/update package as agentlab’s, so the two CLIs behave alike.
Changed
muster version and muster --version print one line, muster version v5.23.5 (commit 361cdef, built 2026-09-15T19:38:51Z), with the details the binary knows; the commit: and built: lines
and their unknown placeholder are gone, and muster version still adds the aggregator’s version
when one is running. A go build from a checkout reports Go’s own stamp – the tag at a tag,
+dirty over local edits, a pseudo-version between tags (v5.23.6-0.20260915195350-977012d023ba:
after v5.23.5, before v5.23.6) – which self-update and the hint compare as what it is; a binary
without any version says dev, and self-update refuses it with a pointer to
go install github.com/giantswarm/muster/v5@latest. The self-update messages follow agentlab’s
(“Nothing newer than v5.23.5 on GitHub (latest release v5.23.5).”; a newer release is announced
with its URL instead of its full release notes), and its logic moved from cmd to
internal/update.
Fixed
muster test: two races of the harness under --parallel 50, both seen once in CI on 2026-09-15.
test_restart_mock_oauth_server listened again on the port the old server had just released, and
any socket on the host could take it in between – the mocks listen on kernel-assigned ephemeral
ports, the range every outgoing connection draws from (failed to listen again on port 45841: bind: address already in use). The mock now keeps its socket bound across the restart and replaces
only the server behind it, as a Service keeps its address while the pod is replaced. And a step
whose call to muster serve never returns (a wait_for_state poll still pending when its budget
ran out; oauth-subject-grant-refresh failed that way with an instance that kept logging and a
harness that kept finishing other scenarios) is reported as stalled instead of “expectations not
met”, and the runner records where everything was: muster serve gets SIGQUIT so its goroutines
end the instance stderr of the report, the harness’s goroutines are stored as harness_goroutines,
and the failure line points at both. The harness also runs the muster binary it is part of before
consulting PATH: a stale go install first on PATH ran the suite against the wrong muster serve.
Fixed
go install github.com/giantswarm/muster/v5@latest builds the release it resolves to. Go refuses
go install <package>@<version> for a module whose go.mod carries a replace directive, and go.mod
carried one since 2026-08-28: goldmark pinned to a fixed version for the nancy scan, because
golang.org/x/tools required a version OSS Index flagged (#1106).
Nothing in the build reaches x/tools or goldmark anymore (go mod why -m github.com/yuin/goldmark:
the main module does not need it; neither is among the packages go list -deps ./... hands to
nancy), so the pin changed nothing but the go install verdict and is gone. The module graph is
unchanged.
Changed
- The Go module is
github.com/giantswarm/muster/v5; every import path carries the suffix
(github.com/giantswarm/muster/v5/pkg/oauth). The module path had no /v5 while the releases were
v5.x, and Go only considers v0 and v1 tags for such a path: go install github.com/giantswarm/muster@latest
resolved to v1.12.0 from August and installed a month-old v1 binary, and
go install github.com/giantswarm/muster@v5.23.2 was refused (“module contains a go.mod file, so module
path must match major version”). go install github.com/giantswarm/muster/v5@latest now resolves to the
newest release, and Go’s own VCS stamping gives a build of a tagged commit its v5 tag instead of a
pseudo-version of the last v1 tag. No repository outside this one imported the old path. The
goimports -local prefix of the pre-commit hook and the contributor docs follow the module. The
OpenTelemetry scope name github.com/giantswarm/muster (observability.TracerName, the otel_scope_name
label, the logger scope) is an identifier dashboards filter on, not an import path, and is unchanged.
Fixed
- The release binaries report their release tag again.
muster version on the v5.22.0 binary printed
v1.12.1-0.20260915144925-e6c760a32b48: the CI build stamped the commit and the build time but no
version, so the binary fell back to the version Go’s own VCS stamping had derived – and as the
module path is github.com/giantswarm/muster with no /v5 suffix, Go only considers v0 and v1 tags
and turned every v5 release commit into a pseudo-version of the last v1 tag. Because that version
is older than every release, muster self-update re-installed the very release it was running on
each time instead of reporting “Current version is the latest.” The build now links the tag into
the binary (make stamp-version, run by make test between the architect orb writing its link
flags and linking with them; a branch build gets git describe, e.g. v5.23.2-1-g4be8379e), a
pseudo-version from the build info is no longer shown as the version, and self-update refuses a
version that is not a release (dev, a bare commit) instead of panicking on it.
(#1256)
Added
- Homebrew: the release pipeline tells the tap
giantswarm/homebrew-muster about a release once
its binaries are on the GitHub Release (a muster-release repository dispatch from
.circleci/custom.yml); the tap’s workflow verifies every binary against its Sigstore bundle
before it regenerates the formula. brew install giantswarm/muster/muster follows each release
again; the tap had been stuck at v0.7.5 since June.
