1. Docs
  2. Changes and releases
  3. AI Agents

AI Agents

  • AI Agents klaus v0.1.22

    Changed

    Full Changelog: https://github.com/giantswarm/klaus/compare/v0.1.21...v0.1.22

  • AI Agents klausctl v0.4.8

    Fixed

    Full Changelog: https://github.com/giantswarm/klausctl/compare/v0.4.7...v0.4.8

  • AI Agents agent-platform v4.32.1

    Fixed

    • vLLM’s cache is a subPath of the claim’s own — .vllm-cache at the claim root, mounted at /mnt/vllm-cache on the runtime container with VLLM_CACHE_ROOT set in the same rule — and no longer a path under the model’s directory (giantswarm/agent-platform#541). 4.31.0 set VLLM_CACHE_ROOT=/mnt/models/.cache/vllm through modelServing.policies.env, and every cold start crash-looped: the storage-initializer’s Hugging Face client creates <model>/.cache/huggingface as uid 1000, mode 755, during the download, so .cache exists before vLLM starts, owned by uid 1000 and not group-writable, and the runtime (another uid, gid 1000 through the claim’s fsGroup) failed os.makedirs('/mnt/models/.cache/vllm') with PermissionError — Engine core initialization failed; a model whose weights were already cached hit the same directory. The redirect rule now mounts the claim a second time on the runtime container of both pod shapes, at /mnt/vllm-cache from the claim-wide subPath .vllm-cache (the kubelet creates a missing subPath directory with the claim root’s group and mode, root:1000 2775, which gid 1000 writes; compiled graphs are keyed by model and configuration inside it, so one directory serves every model), and sets VLLM_CACHE_ROOT=/mnt/vllm-cache on that container in the same patch — the env is never set without the directory behind it, and a pod without the cache keeps vLLM’s default cache root. modelServing.policies.env is back to HF_HUB_DISABLE_XET alone in both charts, and the runtime’s /mnt/models mount keeps the attributes KServe declares again (the classic predictor’s read-only): the runtime writes nothing into the model’s directory. make verify-model-serving-policies asserts the cache mount and the env together on the runtime container of both shapes and on the storage-initializer neither, the model mount’s readOnly unchanged, that no mount or env value of the pod names a path under /mnt/models, and that a workload pod without a model name gets neither the mount nor the env. An installation on 4.31.0 with a crash-looping model: the next pod after the upgrade starts clean; the <model>/.cache/huggingface directory the initializer created stays and is harmless.
  • AI Agents agent-platform v4.32.0

    Added

    • The connectivity chart ships the PriorityClass the gpu-node-pool chart’s prewarm placeholder runs at (giantswarm/agent-platform#539). A pool created with prewarm: true launches its first node with the release: the pool chart’s one-shot placeholder Job holds one GPU at negative priority until the first predictor preempts it. A PriorityClass is cluster-scoped, and a pool release is delivered under Flux multi-tenancy as the organisation’s tenant ServiceAccount, whose rights are namespaced — a pool chart that rendered the class failed InstallFailed on it and the pool never came up. The connectivity chart now renders PriorityClass agent-platform-prewarm-placeholder — value: -1000 (below the default priority 0, so any workload preempts the placeholder), preemptionPolicy: Never, globalDefault: false, the cluster-manager component label, a description naming its use — from clusterManager.prewarmPriorityClass (enabled: true, name, value), mirrored in the meta chart; the pool chart’s pool.prewarm.priorityClassName defaults to the name and renders no class of its own (giantswarm/gpu-node-pool#15). The class renders with the cluster-manager component, which composes the pool releases and is on in exactly one release per installation (the platform’s) — not with the model-serving switch, which on an installation is on in the serving slice cluster-manager creates beside the pool: a class gated there would arrive after the pool’s Job and leave with the slice. enabled: false renders none (a placeholder pod naming a class the cluster lacks is rejected at admission; the pool itself comes up without a prewarmed node); value is immutable on the API, so a change is a rename. Guards fail the render naming the key on a value at or above 0 or fractional, a name that is not a DNS-1123 subdomain, and the reserved system- prefix. tests/verify-mirrored-values.py holds the block equal between the charts (a forwarded copy under another name would rename the class every pool expects), with a negative control in make verify-meta; make verify-cluster-manager asserts the class, its knobs, the guards and that nothing renders while the component is off. README: “The prewarm placeholder’s PriorityClass”.
  • AI Agents agent-platform v4.31.0

    Added

    • The model-serving cache outlives a pool teardown, its claim gets a fast StorageClass by default, and vLLM’s compile cache lives on the claim (giantswarm/agent-platform#537). Measured on a GPU pool scaling from zero: every new pool downloaded the weights again (the Helm-owned serving namespace took the kept hf-cache claim down with it on uninstall), the weights loaded at the default gp3 baseline (8.8 GiB in 71 s = 124 MiB/s, the class’s cap), and torch.compile ran on every pod (its cache in an emptyDir). The connectivity chart now: (1) marks the serving namespace helm.sh/resource-policy: keep while modelServing.cache.enabled is on — an uninstall leaves the namespace, the claim and its volume behind, nothing else; a re-install adopts them; (2) renders a StorageClass of the claim’s own (modelServing.cache.storageClass, default create: true: <chart>-<claim> = agent-platform-connectivity-hf-cache, provisioner: ebs.csi.aws.com, gp3 at throughput: "1000" / iops: "4000", WaitForFirstConsumer, expansion allowed, reclaimPolicy: Delete — the claim is the durable object) and the applied claim references it; create: false with name references an existing class and renders none; pvc.storageClassName next to create: true is refused naming both; the default is AWS’s, another cloud sets provisioner and parameters; (3) adds VLLM_CACHE_ROOT=/mnt/models/.cache/vllm to modelServing.policies.env and mounts the runtime container’s /mnt/models read-write in both pod shapes, so a model’s compiled graphs persist in its cache directory. The meta chart mirrors every new default; tests/verify-mirrored-values.py now holds the whole modelServing.cache and modelServing.policies blocks equal between the charts (a forwarded env without the entry would drop it from every installation), with a second negative control in make verify-meta. An installation whose claim already exists on another class keeps it with storageClass.create: false and name: <its class> (storageClassName is immutable; the hook fails naming the field otherwise).
  • AI Agents agent-platform v4.30.3

    Fixed

    • Preset qwen3-8-27b declares the Hub’s weight size, 25 GiB (was 15), and names the hardware it is tuned for; every shipped preset’s weightsGiB is checked against the Hub in CI (giantswarm/agent-platform#535). Two components judge whether a GPU pool hosts a preset from two inputs: cluster-manager’s create_node_pool sizes the pool from the preset’s declared weightsGiB + overheadGiB, model-manager’s check_fit / load_model size the weights from the Hub (model.safetensors.index.json total_size, else the safetensors files) and add the preset’s overheadGiB. qwen3-8-27b declared 15 GiB for a 24.57 GiB repository (Inferact/Qwen3.8-27B-NVFP4: six shards plus nvfp4_experts_mtp.safetensors), so a pool of one 48 GiB GPU was judged to host it on the form (45 GiB) and refused at the serve step (54.6 GiB). The preset now declares 25 GiB and its description says that, as tuned (25 GiB of weights + 30 GiB of overhead, the CUTE_DSL_ARCH pin, --gpu-memory-utilization=0.60), no size of the curated AWS accelerator families hosts it — its target is a single 128 GB unified-memory Blackwell node; the flags are unchanged. devstral-small-2 declared 48 GiB “at BF16” for a 24.02 GiB FP8 checkpoint (mistralai/Devstral-Small-2-24B-Instruct-2512: quant_method: fp8; the index totals the six model-* shards, the consolidated-* files are a second copy of the same weights) and now declares 25 GiB. New make verify-preset-weights (tests/verify-preset-weights.py; collected by verify-all, so CI runs it for every PR) sizes each preset’s spec.model.id the way model-manager does and fails a preset below the Hub’s size, more than 15 % above it, or one the Hub cannot size — never skipped; tests/fixtures/serving-preset-weights-{understated,overstated}.yaml are its negative controls. Every other shipped preset is within tolerance (+0.3 % to +6.9 %).
  • AI Agents muster v5.27.1

    Fixed

    • A live session follows its MCPServer’s auth configuration. When an MCPServer’s spec.auth changed underneath sessions connected to it – forwardToken replaced by a pinned authorizationServer, another issuer, expectedIssuer, grant scope or client Secret, a changed token exchange – every live session kept its state for the server from before the change: muster auth status said Connected, a tool call failed with the backend’s 401 as Tool execution failed: failed to call tool: transport error: authorization required, muster auth login --server <name> refused (already connected and does not require authentication), and nothing short of signing out of muster ended it, while the server itself read Auth Required all along. Three things changed. The aggregator puts the server back to auth_required for every live session when the changed configuration registers (session_auth_reset in the log; a restart or a retry on an unchanged configuration leaves the sessions be); the session’s cached tools stay resolvable, so its next call answers with the sign-in. A tool call the session cannot make – it is not signed in to the server, or the backend refuses its credential with a 401 during the call – answers the auth_required challenge with the sign-in link for that server, the answer core_auth_login gives (marked as an error: the tool did not run; structuredContent.authUrl carries the link), never user not authenticated to server or a transport error; the refused connection is retired on that call, without waiting for the transport’s three retries, and an SSO server’s answer says to sign in to muster again. The streamable HTTP transport’s bare 401 (authorization required) is now recognized as one wherever muster tests for a 401. And muster auth logout --server <name> signs the session out of that one server at the aggregator (core_auth_logout) and prints what happened to the person’s grant; it used to print guidance and change nothing, and --server next to --endpoint signed out of the aggregator instead. Scenarios oauth-auth-config-change-resets-live-sessions and oauth-backend-401-answers-auth-required; test_pin_mcpserver_authorization_server drops forwardToken/tokenExchange when it pins. (#1276)
  • AI Agents agent-platform v4.30.2

    Fixed

    • (connectivity) Presets quote their JSON arguments for the llm-d runtime’s shell re-parse; the chart refuses an argument the shell would re-split in #533 by @teemow

    Full Changelog: https://github.com/giantswarm/agent-platform/compare/v4.30.1...v4.30.2

  • AI Agents klaus-oci v0.0.75

    Fixed

    Full Changelog: https://github.com/giantswarm/klaus-oci/compare/v0.0.74...v0.0.75

  • AI Agents muster v5.27.0

    Added

    • spec.auth.authorizationServer.expectedIssuer: two clients of one authorization server. A pinned authorization server’s issuer was both the identity the grant is filed under and the value the RFC 9207 iss parameter of the authorization response is compared against. GitHub sends iss=https://github.com/login/oauth on every response, so a GitHub App pinned under its own identity (https://github.com/apps/<slug>, to keep its grant apart from another App’s) never completed its sign-in: iss mismatch: response carried "https://github.com/login/oauth", expected "https://github.com/apps/<slug>". The new optional field names the issuer identifier the server really publishes; a present iss is compared against it (a trailing slash tolerated) while issuer stays the grant key, so each App keeps its own grant, consent and core_auth_logout. It needs the pinned endpoints; with the field unset nothing changes. Carried through the CRD, the API, core_mcpserver_get and the aggregator’s pin; the how-to gained “Two clients of one authorization server” and the CRD reference the pin’s fields. (#1277, #1275)