AI Agents

• Sep 18, 2026 AI Agents muster v5.27.2

  Fixed

  • A remote server’s spec.timeout governs each tool call. The value bounded the first connect, the health probe and the session-recovery handshake, while the requests themselves – a tool call above all – ran under the caller’s context alone: a tool that legitimately blocks for two minutes behind a server declaring timeout: 180 was cut whenever the caller’s connection gave up first, and muster reported the cut as a transport failure of the POST (transport error: failed to send request: Post "http://..."). Now every operation on a remote client – each request, and a recovery handshake on the way – runs under the server’s spec.timeout, the CRD’s default of 30 s when unset, so the documented range (1-300) holds end to end; a call that outlasts the budget fails with no answer within the server's timeout of 60s: context deadline exceeded rather than a transport error. The session-scoped clients of servers that need a sign-in get the same budget through their registration. (#1281)
• Sep 18, 2026 AI Agents agent-platform v4.38.1

  Changed

  • (golden) Drop the holds GOLDEN_REF now carries — every one only narrowed the comparison, and three of them compared a shape no installation gets in #531 by @fiunchinho

  Full Changelog: https://github.com/giantswarm/agent-platform/compare/v4.38.0...v4.38.1

• Sep 18, 2026 AI Agents agent-platform v4.38.0

  Added

  • Preset qwen3-8-flash-next-nvfp4: Qwen3.8-Flash-Next NVFP4 on one 128 GB unified-memory Blackwell node, served from an OCI model image on an installation-defined runtime (giantswarm/agent-platform#553). Qwen3.8-Flash-Next (125B total, 6B active, a 51B n-gram embedding table, 262k context, tools, reasoning, vision) had no shipped preset: NVIDIA’s NVFP4 checkpoint is 123.6 GiB on disk and does not fit the node next to a KV cache. The new preset serves the quantization-aware-distilled repack local-inference-lab/Qwen3.8-Flash-Next-NVFP4 (98.6 GiB, the n-gram table in NVFP4 too) with the community’s single-node recipe: the table read from disk (VLLM_PLE_TABLE_MEMORY=disk), MTP speculative decoding (--speculative-config='{"method":"mtp","num_speculative_tokens":4}'), the B12X load format, GDN decode kernel, linear and MoE backends, --quantization=modelopt_mixed, --kv-cache-dtype=fp8, --mamba-cache-mode=align, prefix caching, chunked prefill, --max-model-len=262144, --max-num-seqs=8, --max-num-batched-tokens=4096, --gpu-memory-utilization=0.80, the qwen3 reasoning parser, tools on with the qwen3_xml parser, fused activation quantization (--compilation-config='{"pass_config":{"fuse_act_quant":true}}'); nine runtime knobs (CUTE_DSL_ARCH=sm_121a, SAFETENSORS_FAST_GPU, VLLM_WORKER_MULTIPROC_METHOD=spawn, VLLM_SSM_CONV_STATE_LAYOUT=DS, the AOT compile flags, VLLM_USE_V2_MODEL_RUNNER, B12X_POLICY_MODE=auto) plus HOME, HF_HOME and VLLM_CACHE_ROOT under /tmp, because the modelcar path runs the runtime as a non-root uid on a read-only image; requests 4 vCPU / 48 GiB, limits 8 / 64 GiB; weightsGiB: 99 (the Hub’s size), overheadGiB: 20 (the table stays on disk, so about 74 GiB are resident and the rest of the 0.80 budget is the KV cache); predictor.timeout: 1800; licence other (the Qwen community licence). Two firsts among the shipped presets: spec.runtime: kserve-vllm-b12x names an additional runtime the installation defines from the B12X image through modelServing.additionalRuntimes (giantswarm/agent-platform#550) — the default upstream runtime cannot serve this architecture, and without the entry the InferenceService names a runtime that does not exist — and spec.model.storageUri is oci://gsoci.azurecr.io/giantswarm/models/qwen3-8-flash-next-nvfp4:7c4f1bc1a2d6 (the tag the first 12 characters of the checkpoint revision), the model as an OCI image a registry holds under that path, its host swapped by modelServing.modelImages.registry and pre-pulled through modelServing.prepull.modelPresets (giantswarm/agent-platform#551). make verify-wiring expects eleven shipped presets; verify-preset-weights and the eval check of verify-serving-slice cover the new file; verify-gpu-pool leaves the preset’s ConfigMap and its discovery entry out of the golden comparison while origin/main predates it.
• Sep 18, 2026 AI Agents agent-platform v4.37.0

  Added

  • Models as OCI images: a registry override for oci:// presets and pre-pull of the named presets’ images on the serving nodes (giantswarm/agent-platform#551). A preset’s weights came from Hugging Face into the cache claim, and the claim binds one node (a local volume) or one zone (a cloud disk): a model served only where the claim was, every other GPU node or zone downloaded it again, and nothing verified what arrived. KServe’s modelcar path — spec.model.storageUri: oci://<registry>/<path>[:tag|@digest], the model as an OCI image containerd pulls onto the node’s local disk, no storage-initializer, no claim — was admitted by the preset schema and left alone by the cache redirect rule, but nothing pointed a preset’s image at an installation’s registry and the pre-pull DaemonSet knew only runtime images. modelServing.modelImages.registry (empty: references as written) swaps the registry host of every oci:// preset at render, a shipped preset’s and a values preset’s alike — the segment before the first /, the path, tag or digest kept — inside the preset resolution, so the preset ConfigMap, the pre-pull DaemonSet and every consumer read one value; hf:// presets are untouched; a registry with a scheme or a path, and an oci:// reference without a host, fail the render naming it. The discovery ConfigMap publishes the host as spec.modelImages.registry. modelServing.prepull.modelPresets names the presets whose images join the pre-pull DaemonSet’s init containers after the runtime images — pull-model-<preset> on the published reference minus oci://, /bin/true (a modelcar image is busybox-based), the runtime init containers’ security context and resources — so every serving node holds the weights before the first serve and the running pause pod keeps the image against the node’s image garbage collection; a name that is not a published preset, is not oci:// or is listed twice fails the render naming it. The meta chart mirrors both blocks (make verify-meta holds every leaf equal); make verify-model-images (new, in verify-all) holds the rewrite over the fixture ci/test-model-serving-oci-values.yaml (an oci:// and an hf:// values preset, every shipped preset by its own scheme, an empty registry as written, a digest kept), the init containers, the guards and the meta chart’s forwarded render; make verify-model-serving-policies holds the pod with a model init container to the fleet’s restricted PSS with no exception. README: “Models as OCI images” — the modelcar contract (/models in a busybox-based image, the runtime’s uid and the /tmp paths an OCI preset sets, what the cache redirect does not do for these pods).
• Sep 18, 2026 AI Agents agent-platform v4.36.0

  Added

  • modelServing.additionalRuntimes: more vLLM ClusterServingRuntimes beside the default, rendered from the same template (giantswarm/agent-platform#550). modelServing.runtime rendered exactly one runtime and every preset ran on it; an installation whose accelerator needs a different serving image for some models — a hardware-specific vLLM build for a unified-memory Blackwell node next to the upstream image the other presets run on — had to move every preset to that image or hand-write a runtime outside the chart. The new list renders one ClusterServingRuntime per entry: name required, a DNS-1123 subdomain and unique across the default and the list (a name already used fails the render naming the entry); every other field — image (registry, name, version), args, env, resources, shmSize, startupProbe, annotations, supportedModelFormats, nodeSelector, tolerations — the default runtime’s where the entry leaves it unset (a mapping field by field, a list whole: an entry’s env replaces the default’s two entries rather than extending them; an empty value inherits too); the GPU pool’s taint and label applied as on the default; the container on the same shell entrypoint (/bin/sh -c 'eval "exec vllm serve $@"' --) with the entry’s own base arguments, so its image must carry /bin/sh and vllm on PATH as well. A preset selects a runtime with spec.runtime. The discovery ConfigMap publishes every runtime’s name as spec.runtimes (the default first) next to spec.runtime; an empty list renders the default alone, and the default runtime’s document is unchanged. Mirrored in the meta chart, and tests/verify-mirrored-values.py (make verify-meta) now holds the modelServing.runtime and modelServing.additionalRuntimes blocks equal between the charts too — an additional runtime inherits from the forwarded default. make verify-runtimes asserts the render over ci/test-model-serving-runtimes-values.yaml (inheritance, the pool on every runtime, the entrypoint, the discovery list, the guards, the meta chart’s forwarded render); README “Additional runtimes”.
• Sep 18, 2026 AI Agents agent-platform v4.35.0

  Added

  • Model and runtime image signatures are verified at admission: modelServing.imageVerification renders a Kyverno verifyImages ClusterPolicy over every model pod of the serving namespace (giantswarm/agent-platform#552). A model pod ran whatever the registry served under its references — the modelcar sidecar that carries the weights, the pre-fetch init container KServe runs from the same image and the runtime container were admitted unverified — and an installation that curates its models as signed OCI images had no way to have the platform refuse an unsigned or tampered one. Off by default; on, the connectivity chart renders <release>-model-serving-image-verification where the cache policies render (Kyverno served) and nowhere else: one rule per pod shape — the classic InferenceService predictor and the LLMInferenceService workload pod, Pods at CREATE and UPDATE, since a pod’s container images are mutable and a CREATE-only rule would admit an image swapped into a running pod — verifying every container image that matches images (Kyverno’s imageReferences, wildcards as Kyverno reads them: gsoci.azurecr.io/giantswarm/models/* covers every curated model image) against attestors, one entry per trusted signer passed through as written into one attestor set with count: 1 (keyless — issuer and subject or subjectRegExp, Rekor — a public key or certificates), mutateDigest: true pinning the verified tag to its digest, required: true, failureAction: Enforce (or Audit). Enabled with an empty images list, no attestor, an entry that is no Kyverno attestor entry, a failureAction outside the two or an unknown key fails the render naming the key. The meta chart mirrors the block (make verify-meta holds every leaf equal); make verify-model-serving-policies asserts the rendered rule of both shapes, runs the policy through kyverno apply over both fixture pods (every rule skipped, the policy accepted by Kyverno’s schema, a misspelt field refused), nothing while disabled or without Kyverno, the guards, and the meta chart’s forwarded render equal. README “Verifying model images”.
• Sep 18, 2026 AI Agents agent-platform v4.34.1

  Fixed

  • The classic vLLM runtime runs its container through the llm-d template’s argument grammar, so every shipped preset serves on the classic InferenceService path too (giantswarm/agent-platform#549). The chart’s kserve-vllm ClusterServingRuntime — the path model-manager composes into wherever the LLMInferenceService API is not served — put a preset’s arguments straight into argv, while the presets write every JSON-valued vLLM flag single-quoted inside one argument (--limit-mm-per-prompt='{"image": 1, "video": 0}', giantswarm/agent-platform#532) because the llm-d runtime template re-parses its arguments with a shell; vLLM received the single quotes literally and refused the value (invalid loads value), so the six presets with such a flag could not start on the classic path, and a bare JSON that would have worked there fails the chart’s argument guard. The runtime container now has the command /bin/sh -c 'eval "exec vllm serve $@"' -- ahead of its base arguments (modelServing.runtime.args, unchanged: --model /mnt/models --port 8080 --served-model-name {{.Name}}, KServe still expands {{.Name}} before the container starts); KServe appends the preset’s arguments after them, so $@ is base + preset and a single-quoted JSON value reaches vLLM as one word with the quotes gone — exactly as on the llm-d template, one grammar for presets. The image must carry /bin/sh and vllm on PATH (the upstream vLLM images do; an installation’s own modelServing.runtime.image must too). make verify-serving-slice now also runs the RENDERED classic command over the rendered base arguments and every shipped preset’s, with a vllm stub dumping argv, and asserts the base arguments (the {{.Name}} placeholder expanded the way KServe does, an installation’s extra ones) and the preset’s reach vLLM one word each with every JSON value parsing on both paths, so the two cannot drift again; the fixture pod of verify-model-serving-policies carries the command. Not changed: a preset’s argument rule (single-quote a value with whitespace or JSON; the render refuses the rest), the base arguments, the port, the image.
  • The shipped presets serve on the llm-d runtime: a JSON-valued vLLM argument is one argument with the JSON single-quoted inside it, and the chart refuses a preset argument the runtime’s shell would re-split (giantswarm/agent-platform#532; proof 1 of giantswarm/giantswarm#37639). The well-known runtime template’s entrypoint (kserve-config-llm-template, from the kserve-runtime-configs chart) ends in eval "exec vllm serve … $@": every argument is re-parsed by the shell, so --default-chat-template-kwargs followed by {"enable_thinking": false} reached vLLM as {enable_thinking: and false} — invalid loads value, CrashLoopBackOff — and qwen3-8b-fp8 could not be served; qwen3-14b, qwen3-5-27b, qwen3-5-35b-a3b, nemotron-3-super-nvfp4 and qwen3-8-27b (three such arguments) carried the same shape. The six presets now write each such value as --flag='{…}', which the shell’s re-parse yields as one word with the JSON intact. The connectivity chart’s resolvePreset fails the render of any preset — shipped or modelServing.presets — whose argument carries whitespace, a quote or a shell metacharacter outside single quotes, naming the preset and the argument; the preset schema’s spec.args states the contract. make verify-serving-slice runs the template’s exact eval over every shipped preset’s arguments (argv dumped in place of vLLM) and asserts one word per argument and a parsing JSON value, and that seven bad argument shapes fail the render naming the guard. The classic InferenceService path gets the same grammar in giantswarm/agent-platform#549 (below).
• Sep 18, 2026 AI Agents agent-platform v4.34.0

  Added

  • The serving runtime image is pre-pulled on the pool’s GPU nodes at join (giantswarm/agent-platform#545). A predictor pulls its runtime image only when its main container starts, after the storage-initializer has finished the weights, so on a pool scaling from zero the two longest steps of a cold start ran one after the other — measured on a fresh node: weights 116 s, then 115 s for the 8.8 GB llm-d-cuda image, while with prewarm the node had been Ready for two minutes. modelServing.prepull (on by default) renders a DaemonSet in the serving namespace with one /bin/true init container per image of prepull.images — the runtime image of the well-known LLMInferenceServiceConfig, gsoci.azurecr.io/giantswarm/llm-d-cuda:v0.8.0, pinned here because the connectivity chart has no other value for it and following the kserve-runtime-configs line, never ahead of it — and a pause main container (prepull.pauseImage, prepull.resources): the kubelet pulls the images the moment the node joins, in parallel with the weights, and containerd keeps them for the predictor. It runs on the nodes prepull.nodeSelector (Karpenter’s karpenter.k8s.aws/instance-gpu-manufacturer: nvidia) and the pool’s own label select, tolerates the pool’s taint first and every taint after it (prepull.tolerations, so the pull starts under a node’s start-up taints), holds no GPU, mounts no token, is denied all traffic by a policy of its own in both flavours (the pull is the kubelet’s, not the pod’s) and passes the fleet’s restricted Pod Security Standard with no exception. The meta chart mirrors the block (make verify-meta holds every leaf equal); make verify-model-serving-policies holds the rendered pod to the fleet’s restricted-PSS policies, the deny-all’s selection and the meta chart’s forwarded render.
• Sep 18, 2026 AI Agents agent-platform v4.33.0

  Added

  • Preset qwen3-8-27b-l40s: Qwen3.8 27B NVFP4 on one 48 GB GPU (giantswarm/agent-platform#544). The shipped qwen3-8-27b is tuned for a 128 GB unified-memory Blackwell node and hosts on no size of the curated AWS accelerator families (giantswarm/agent-platform#535). The new preset serves the same Inferact NVFP4 repack on one L40S — g6e.2xlarge or g6e.4xlarge — with the flags a live probe proved: --gpu-memory-utilization=0.92, --max-model-len=65536, --max-num-seqs=4, --max-num-batched-tokens=8192, chunked prefill, prefix caching, one image per prompt, the qwen3 reasoning parser, tools on with the qwen3_coder parser, thinking off by default, no speculative decoding, no CUTE_DSL_ARCH, the model’s own chat template; requests 4 vCPU / 32 GiB, limits 8 / 48 GiB; weightsGiB: 25 (the Hub’s size), overheadGiB: 16; predictor.timeout: 1800. Ada has no native FP4: vLLM takes the Marlin weight-only FP4 path, loads 24.17 GiB of weights and keeps 14.65 GiB for the KV cache (222,822 tokens, 3.4 concurrent 64k requests). cluster-manager’s presetFit hosts it on 2xlarge and up of the L40S family (g6e.xlarge leaves a predictor 3 vCPU / 27.1 GiB) and on no L4, A10G or T4 size (25 + 16 = 41 GiB > 24). make verify-wiring expects ten shipped presets; verify-preset-weights and the eval check of verify-serving-slice cover the new file.
• Sep 18, 2026 AI Agents klaus-operator v0.0.163

  Fixed

  • (deps) Update module github.com/giantswarm/klaus-oci to v0.0.75 in #214 by @renovate[bot]

  Full Changelog: https://github.com/giantswarm/klaus-operator/compare/v0.0.162...v0.0.163

• Newer entries
• Older entries