Last modified May 25, 2023
Handling cloud provider credentials
In order to manage workload clusters in your cloud provider accounts/subscriptions, the Giant Swarm controllers require some configuration so they are able to act on your behalf.
In this article we explain how to provide this configuration via the Management API.
As we are dealing with two cloud providers here which use different vocabulary in their own worlds, let’s first establish some terminology for this article.
- Account (AWS) or Subscription (Azure): we use the term account here for both.
- Credentials: a set of details which enable a controller or a Giant Swarm staff member to work with your cloud provider account on your behalf. This article is mostly about providing these credentials in the right place and format.
Credential secrets explained
Credentials are stored in the management cluster in the form of a
metadata, credential secrets must provide the
giantswarm.io/managed-by: credentiald and
app: credentiald labels. We also recommend to choose a resource name starting with
data part contains the cloud provider credentials that should be used. The format here differs slightly between cloud providers.
Here is an example for AWS:
apiVersion: v1 kind: Secret type: Opaque metadata: name: credential-example namespace: my-namespace labels: app: credentiald giantswarm.io/managed-by: credentiald data: aws.admin.arn: YXJuOmF3czppYW06OjEyMzQ1Njc4OTA6cm9sZS9HaWFudFN3YXJtQWRtaW4= aws.awsoperator.arn: YXJuOmF3czppYW06OjEyMzQ1Njc4OTA6cm9sZS9HaWFudFN3YXJtQVdTT3BlcmF0b3I=
In above AWS example, these are two fields which both indicate AWS IAM role identifiers (ARNs). As typical with opaque secrets, the value is encoded in base64. Here is how the two example strings would look like when decoded:
|Key||Example value (decoded)|
The first of the two identifies the IAM role to be assumed by Giant Swarm staff for operations tasks. The second one is used by
aws-operator, which is the software in charge of automatic cluster management. Easily visible, they include the numeric AWS account ID and also the IAM role name to be assumed.
We provide detailed documentation regarding how to configure these roles in your AWS account.
Here is an example for a credential secret on Azure:
apiVersion: v1 kind: Secret type: Opaque metadata: name: credential-example namespace: my-namespace labels: app: credentiald giantswarm.io/managed-by: credentiald data: data: azure.azureoperator.clientid: MTIzNGFiY2QtYWIxMi0xMmFiLWNkMzQtYWJjZDEyMzRhYmNkCg== azure.azureoperator.subscriptionid: NTY3OGFiY2QtYWI1Ni01NmFiLWNkNTYtYWJjZDU2NzhhYmNkCg== azure.azureoperator.tenantid: MTIzNGRlZmctZGUxMi0xMmRlLWRlMzQtZGVmZzEyMzRkZWZnCg== azure.azureoperator.clientsecret: YWJjZC1lZmdoaWprbG1uTE9QUTEyMzQ1Njd+ODlSU3R1dnd4Cg==
data part contains four fields which specify the IDs of
Tenant as well as the
|Key||Example value (decoded)|
In order for
azure-operator to manage workload clusters using these credentials, it will need access to your
Subscription using a
We provide a detailed guide prepare and obtain this data.
Referencing credentials in the cluster resource
When creating a workload cluster, depending on the cloud provider, there will be an
AzureCluster resource created, containing the provider-specific configuration of your workload cluster.
If you create your cluster manifest manually or via the
kubectl gs template cluster command, you are free to adapt the provider cluster resource to match your exact requirements before submitting the manifest to the API (e. g. via
In AWS, it is possible to reference your credential secret directly. The custom resource definition (CRD) provides an attribute
.spec.provider.credentialSecret. The two sub-attributes
name has to match the credential secret’s resource name. The
namespace sub-attribute accordingly must match the secret’s namespace.
Note that this attribute has to be set before submitting the manifest to the Management API. Otherwise, if the
AWSCluster resource gets submitted without any
.spec.provider.credentialSecret in place, our admission controllers will fill in default values. Read on to understand the defaulting logic.
On Azure, the
.spec.identityRef is set by the
azure-operator. It references credential secrets indirectly by creating an
org-credential based on the organization your workload cluster is created in.
If the cluster resource does not reference a provider credential secret to use (as explained above), our admission controller fills in some defaults, in this logical order
- An organization’s credential secret is used, if it exists
- Otherwise the installation’s default secret is used
Both options are explained in further detail below.
Organization default credentials
Any workload cluster in Giant Swarm belongs to an organization as encoded in the
giantswarm.io/organization label of the cluster’s
For each organization you can also deposit a credential secret which will be used automatically for any new workload cluster that does not specify credentials explicitly (as explained above). To be looked up as the organization’s default credentials, your
Secret resource must match the following requirements:
- The label
giantswarm.io/organizationmust be present, with the value matching the organization name
- There must not be any other credential secret matching the same
This partial example shows the required metadata:
apiVersion: v1 kind: Secret type: Opaque metadata: name: credential-example namespace: org-example labels: app: credentiald giantswarm.io/managed-by: credentiald giantswarm.io/organization: acme data: ...
Installation default credentials
As a last resort, if credentials are not referenced explicitly and no organization default credentials are found, our defaulting will use the installation default credentials.
The installation default credentials are typically configured for you by Giant Swarm, using credentials that you provide. They are placed in the namespace
giantswarm and thus normally not accessible to customers directly.
The idea of the default credentials is to enable every organization to create clusters, even if they don’t have specific credentials configured. This can be a viable use case if all tenants are trusted and should be allowed to use the same cloud provider account.
Choosing the right method for you
The defaulting options as well as the option to reference credentials explicitly per workload cluster should offer the right method for each use case. To sum up the different methods and reasons to use them:
If you want to run all workload clusters in the same cloud provider account, use the installation default credentials. As a result you won’t have to worry about this detail during cluster creation at all.
If you plan to run each organization’s clusters in a different cloud provider account, use the organization default credentials mechanism. Just make sure to set up the organization’s credentials as early as possible, to prevent any cluster running via the installation default credentials.
On AWS, you can set the cloud provider credentials explicitly for more flexibility. Whether you want to rely on organization level or installation level defaults as a fall back is up to you.
- aws-admission-controller: Explore the source code of the component that does the defaulting of AWS credentials on cluster creation.
- azure-operator: Explore the source code of the component that takes care of the indirect reference of Azure credentials on cluster creation.
Need help, got feedback?
We listen to your Slack support channel. You can also reach us at email@example.com. And of course, we welcome your pull requests!