Configure alerting

Learn how to configure alerting in the Observability Platform.

The Giant Swarm Observability Platform provides an alerting pipeline that you can configure per tenant. This tutorial explains how to configure alerting for your tenant.

Prerequisites

Before you begin, make sure you have:

• Access to your management cluster
• A tenant defined in a Grafana Organization

Configure Alertmanager

The Observability Platform uses Mimir Alertmanager for alerting capabilities.

To configure Alertmanager for your tenant, create a secret on your management cluster with the required labels and configuration data.

Example configuration

The following example shows an Alertmanager configuration for the myteam tenant:

apiVersion: v1
kind: Secret
metadata:
  labels:
    # This label marks this secret as an Alertmanager configuration
    observability.giantswarm.io/kind: alertmanager-config
    # This label specifies the tenant for this configuration
    observability.giantswarm.io/tenant: myteam
  name: my-alertmanager-config
  namespace: default
stringData:
  # This field is mandatory and contains the Alertmanager configuration
  alertmanager.yaml: |
    route:
      receiver: "example_receiver"
      group_by: ["example_groupby"]
    receivers:
      - name: "example_receiver"
    templates:
      - example_template.tmpl    
  # Optional: Include reusable template files with .tmpl extension
  # See: https://prometheus.io/docs/alerting/latest/notification_examples/#defining-reusable-templates
  example_template.tmpl: |
    {{ define "alert_customer_env_message" }}
    [{{ .CommonLabels.alertname }} | {{ .CommonLabels.customer }} | {{ .CommonLabels.environment }}]
    {{ end }}    
type: Opaque

Configuration validation

Validate your Alertmanager configuration before applying it. A validating webhook rejects invalid configurations.

Recommendation: Use mimirtool in your CI pipeline to validate configurations before deployment.

Important considerations

• Tenant validation: The observability.giantswarm.io/tenant label must reference a tenant defined in an existing Grafana Organization
• Template files: The platform processes only files with the .tmpl extension as templates
• Webhook validation: The platform automatically rejects invalid configurations or non-existent tenants

Configuration limitations

The Observability Platform uses Mimir Alertmanager, which may not support all the latest Alertmanager features available in the upstream Prometheus Alertmanager. Some configuration options might not work as expected or may not be available.

While the validating webhook ensures basic configuration correctness, we recommend testing your Alertmanager configuration in a non-production environment first. The webhook validates syntax and structure but cannot guarantee that all configuration options work as expected with Mimir Alertmanager.