WWarnFireDocs

Webhook adapters

Prometheus Alertmanager

Point Alertmanager at WarnFire in one config block — alerts open incidents, recoveries close them.

If your team uses Prometheus, Alertmanager is the part that decides where alerts go. This page points it at WarnFire, so firing alerts open incidents and page your on-call responder — and recoveries close them automatically.

What you need

  • An integration key for the service these alerts belong to. Create one under Configuration → Services (see Create your workspace if you’re starting fresh).
  • Access to your Alertmanager configuration file.

Configure Alertmanager

A practical, complete configuration looks like this:

route:
  receiver: warnfire
  group_by:
    - alertname
    - cluster
    - service
  group_wait: 15s
  group_interval: 5m
  repeat_interval: 4h

  routes:
    - receiver: warnfire
      matchers:
        - severity=~"critical|warning"

receivers:
  - name: warnfire
    webhook_configs:
      - url: https://api.warnfire.com/v1/integrations/webhooks/alertmanager
        send_resolved: true
        max_alerts: 50
        http_config:
          authorization:
            type: Bearer
            credentials_file: /etc/alertmanager/warnfire-integration-key

Keep send_resolved: true. It’s what lets WarnFire close the incident by itself when the problem clears.

Place your WarnFire integration key — just the key, nothing else — in the referenced file:

wf_live_your_integration_key_here

Lock down its permissions so only Alertmanager can read it:

chmod 600 /etc/alertmanager/warnfire-integration-key

The key travels in a header — never put it in the URL, where logs and proxies could capture it.

Then validate your configuration and reload Alertmanager:

amtool check-config /etc/alertmanager/alertmanager.yml
curl -X POST http://localhost:9093/-/reload

Here’s an alert rule written to take full advantage of the mapping — its labels become the incident’s severity, component, environment, and region, and its annotations become the title, description, and links:

groups:
  - name: warnfire-example
    rules:
      - alert: CheckoutHighErrorRate
        expr: |
          (
            sum(rate(http_requests_total{
              service="checkout",
              status=~"5.."
            }[5m]))
            /
            sum(rate(http_requests_total{
              service="checkout"
            }[5m]))
          ) > 0.05
        for: 5m

        labels:
          severity: critical
          service: checkout
          component: checkout-api
          environment: production
          region: us-west-2
          team: payments

        annotations:
          summary: Checkout error rate exceeds 5%
          description: More than 5% of checkout requests have failed for five minutes.
          runbook_url: https://docs.example.com/runbooks/checkout-errors
          dashboard_url: https://grafana.example.com/d/checkout

How your alerts become incidents

Alertmanager sends alerts in groups (that’s its group_by setting). WarnFire turns each group into one incident:

  • When the group is firing, the incident opens and paging starts.
  • When the group is resolved, the same incident is closed. (send_resolved: true is what makes this happen — don’t turn it off.)
  • If the group fires again with new information — a changed description, a new measurement, another alert joining the group — the incident gets an update, not a duplicate. Retries of the same delivery are recognized and ignored, so a flaky network can’t double-page anyone.

Because each group becomes one incident, your group_by choice decides what “one incident” means:

group_by:
  - alertname
  - cluster
  - service

This produces separate incidents for different alerts, clusters, or services. If you group only by alertname, alerts from multiple services or clusters may be combined into a single WarnFire incident — fine for some teams, too coarse for others. Prefer one incident per alert? Make the grouping finer — WarnFire follows whatever grouping you choose.

What shows up on the incident

WarnFire maps your alert data to the incident automatically:

On the incidentComes from
TitleThe summary annotation, or the alert name
SeverityThe severity label — critical, error, warning (or warn), info. Anything else becomes error, with your original value kept in the labels.
DescriptionThe description annotation
LinksYour Alertmanager console, runbook_url and dashboard_url annotations, and each alert’s source graph
DetailsThe group’s member alerts (up to 10 listed), firing/resolved counts
LabelsAll the group’s common labels

Try it before you rely on it

Use the console’s built-in tester to see exactly how a payload will map — and to send a safe test through the whole pipeline. See Test before you connect.

If Alertmanager gets an error back

ErrorWhat it means
401The integration key is missing, wrong, expired, or revoked. Check the credentials file.
400The payload wasn’t valid — most commonly a payload version other than 4, or an alert status WarnFire doesn’t recognize.
429Too many alerts too fast for this key’s rate limit. See Limits and protections.