Last modified April 27, 2020

Labelling tenant clusters

It is possible to assign key value labels to Giant Swarm tenant clusters with release version 10.0.0 and above on AWS.

Labels are a mechanism to assign short pieces of additional information to your Giant Swarm tenant clusters. Under the hood, tenant cluster labels are Kubernetes labels attached to Cluster (clusters.cluster.x-k8s.io) resources. Therefore, all means of listing tenant cluster labels will return all Kubernetes labels attached to Cluster resources requested. Label keys and values are freely modifiable except labels with keys containing giantswarm.io.

Working with tenant cluster labels works likewise as working with Kubernetes labels. More information about Kubernetes Labels can be found in the Kubernetes Labels and Selectors and our cluster labels API documentation.

Working with tenant cluster labels using gsctl

With gsctl, our CLI, cluster labels can be modified by executing gsctl update cluster by setting label changes using one or multiple --label flag.

Deleting a label can be accomplished by setting its value to an empty string. --label labeltodelete=.

$ gsctl update cluster 7g4di --label my-org/team=upstate --label my-org/environment=testing
Cluster '7g4di' has been modified.
New cluster labels:
my-org/environment=testing
my-org/team=upstate

Once clusters are labelled, the output of gsctl list clusters can be augmented by setting flag --selector (or -l) It takes a Kubernetes Label selector to specify requirements on cluster labels to select.

The output of gsctl show cluster will contain all labels currently attached to the selected cluster.

Working with tenant cluster labels using the Giant Swarm API

Tenant cluster labels of clusters with release version 10.0.0 and above on AWS are returned by executing a getClusters request. The field labels of suitable tenant clusters contains the labels currently attached to the cluster. Labels of a single tenant cluster can be retrieved using the getClusterLabels endpoint.

Selecting tenant clusters based on a set of labels can be achieved through the getV5ClustersByLabel operation. The operation accepts label selectors in the same way that kubectl get -l does (Kubernetes Label selectors) for listing clusters based on their labels.

The labels of a tenant cluster can be modified by issuing a setClusterLabels request to the API. Keys and labels should adhere to Kubernetes labels syntax and character set. Label changes should be written as a JSON Merge Patch, RFC 7386. Changes to labels with keys containing giantswarm.io is forbidden, changes to label release.giantswarm.io/version will be validated against available Giant Swarm releases.

Differing from gsctl, listing tenant cluster labels with the API will show management labels required for operation. These usually contain giantswarm.io in its label keys and cannot be changed.

Example

Let’s play through a simple workflow of assigning labels to a newly created tenant cluster and selecting it based on the given label.

For brevity authentication and unrelated parts of requests and responses are left out.

After creation, a tenant cluster will already have some labels containing information about release and operator.

GET /v5/clusters/7g4di
{
  "api_endpoint": "...",
  "create_date": "...",
  "id": "7g4di",
  "master": {...},
  "name": "...",
  "owner": "my-org",
  "release_version": "11.2.0",
  "conditions": [...],
  "labels": {
    "cluster-operator.giantswarm.io/version": "2.1.9",
    "giantswarm.io/cluster": "7g4di",
    "giantswarm.io/organization": "my-org",
    "release.giantswarm.io/version": "11.2.0"
  }
}

In our example, the cluster 7g4di already has four labels (cluster-operator.giantswarm.io/version, giantswarm.io/cluster, giantswarm.io/organization, release.giantswarm.io/version).

The newly created cluster will be managed by your team in your upstate office and is planned to be used for testing purposes.

You’ve decided on using label keys my-org/team and my-org/environment to specify the clusters designation.

PUT /v5/clusters/7g4di/labels/
{
  "labels": {
    "my-org/team": "upstate",
    "my-org/environment": "testing"
  }
}

Another cluster from earlier is also managed by your team in the upstate office but is being used for production.

PUT /v5/clusters/g8s2o/labels/
{
  "labels": {
    "my-org/team": "upstate",
    "my-org/environment": "production"
  }
}

From this point on it is possible to select the clusters by label values or label key existence.

POST /v5/clusters/by_label/
{
  "labels": "my-org/team=upstate"
}

will return all clusters managed by the upstate office team regardless of other label values.

The full documentation about label selectors can be found on the Kubernetes Labels and Selectors page.

Working with tenant cluster labels using kubectl

With access to the control plane, you are able to use kubectl to manage tenant cluster labels. The underlying resource to operate on is clusters.cluster.x-k8s.io from the upstream cluster-api project.

Detailed documentation and examples of kubectl label and other commands used here can be found in the Kubectl Reference Docs.

Modify tenant cluster labels

All means of modifying cluster resources can be used to modify labels of a clusters.cluster.x-k8s.io resource.

Interactively, cluster labels can be modified using kubectl edit. Just edit the metadata.labels property.

kubectl edit clusters.cluster.x-k8s.io/7g4di

It is also possible to modify tenant cluster labels with kubectl patch. More information about kubectl patch is available on the Update API Objects in Place Using kubectl patch page.

kubectl patch clusters.cluster.x-k8s.io/7g4di --type merge -p '{"metadata":{"labels":{"my-org/team":"upstate"}}}'

Additionally, kubectl label can be used to modify tenant cluster labels.

kubectl label clusters.cluster.x-k8s.io/7g4di my-org/team=upstate

Show tenant cluster labels

Differing from gsctl, listing tenant cluster labels with kubectl will show management labels required for operation. These usually contain giantswarm.io in its label keys and cannot be changed.

Labels of all tenant clusters:

$ kubectl labels --list --all clusters.cluster.x-k8s.io
Listing labels for Cluster.cluster.x-k8s.io/7g4di:
 giantswarm.io/cluster=7g4di
 giantswarm.io/organization=my-org
 release.giantswarm.io/version=11.2.0
 cluster-operator.giantswarm.io/version=2.1.9
 my-org/team=upstate
 my-org/environment=testing
Listing labels for Cluster.cluster.x-k8s.io/zv86a:
 cluster-operator.giantswarm.io/version=2.1.9
 giantswarm.io/cluster=q84ct
 giantswarm.io/organization=my-org
 release.giantswarm.io/version=11.3.0
 my-org/team=upstate
 my-org/environment=production

Labels of a single tenant cluster:

$ kubectl labels --list clusters.cluster.x-k8s.io/7g4di
giantswarm.io/cluster=7g4di
giantswarm.io/organization=my-org
release.giantswarm.io/version=11.2.0
cluster-operator.giantswarm.io/version=2.1.9
my-org/team=upstate
my-org/environment=testing

Select tenant clusters by label selector

Many kubectl commands support the -l, --selector flag, which allows to limit the selected resources based on given Kubernetes Label selectors.

kubectl get clusters.cluster.x-k8s.io -l 'my-org/team=upstate'