Prometheus Alertmanager 0.22.0 release

Alertmanager is the tool that dispatches Prometheus alerts. It is a core component of a Prometheus monitoring stack.

One year after the previous release, Alertmanager 0.22.0 has been released.

During this timeframe, we have merged a lot of new features, which I will showcase in this blog post.

New date picker in silence form

The experience creating a new silence is now greatly improved with the addition of a new date picker. To view the date picker, simply click on the calendar icon next to the date fields.

Alertmanager date picker

Note: in Alertmanager, all dates are in UTC.

Negative matchers

Alerts are identified by labels in Alertmanager. In order to decide where to send an alert, matchers are used. Matchers are rules that can match labels with fixed values and regexes.

The matchers are used in the routing, in the silences, and for the inhibition of alerts.

This release adds a revamp of the alertmanager matchers. New matchers can be defined inline, in a PromQL-Like syntax, such as:

route:
  receiver: team-X-mail
  routes:
    - matchers: [ '{severity="critical",environment="prod"}' ]
      receiver: team-X-pager

There is also a short form:

route:
  receiver: team-X-mail
  routes:
    - matchers: [ severity=critical, environment=prod ]
      receiver: team-X-pager

With this new matchers, it is also possible to have negative matchers:

route:
  receiver: team-X-pager
  routes:
    - matchers: [ '{severity!="critical",environment!="prod"}' ]
      receiver: team-X-mail

This provides new ways of writing your alertmanager configuration, which can make them shorter or more readable.

amtool, the alertmanager command line tool, also knows the new matchers. In this snippet, we silence all the alerts which do not have a label “environment” equal to “production”:

amtool silence add --alertmanager.url=http://127.0.0.1:9093 \
  -c 'Silencing everything but production.' \
  'environment!=prod'

The reference documentation provides more details.

Time-based muting

Another great feature is the addition of time-based muting in the routes. It means that alertmanager can decide whether to send notifications based on defined time windows.

route:
 receiver: team-X-pager
 routes:
   - matchers: [ '{severity!="critical",environment!="prod"}' ]
     receiver: team-X-mail
     mute_time_intervals: [out-of-business-hours]

mute_time_intervals:
  - name: out-of-business-hours
    time_intervals:
     # Mute on Saturdays and Sundays, all day.
     - weekdays: ['Saturday','Sunday']
     # Mute in the morning and in the evening, any day.
     - times:
       - start_time: '00:00'
         end_time: '08:00'
       - start_time: '18:00'
         end_time: '24:00'

This configuration means that team-X-mail will only receive emails during weekdays, between 08:00 and 18:00 UTC.

Other enhancements

Alertmanager now supports TLS and basic authentication, like the Node Exporter.

The same flags and configuration format are used for both projects, providing consistency in the way you can secure your Prometheus stack.

Conclusion

This alertmanager release is packed - and there is more to come. Notably, we have lifted the moratorium on new integrations, and we expect new notification integrations to be implemented over the next few months.

You can read the full changelog to learn about all the other bug fixes and enhancements.