Weekday Slack triage digest for firing CloudWatch alarms

Every weekday at 8am, pull my currently firing Amazon CloudWatch alarms, reason about them as a set, and post a triaged digest into a single Slack ops channel. The goal is a short, prioritized morning read, not a raw dump.

Start by calling Amazon CloudWatch's Describe Alarms with StateValue set to ALARM to get every alarm currently in the ALARM state. If nothing is firing, post a one-line 'all clear' message to the Slack ops channel using Slack Bot's Send a Message and stop. Do not invent problems.

For each firing alarm, call Amazon CloudWatch's Describe Alarm History with a 24-hour lookback and item type set to state update. Use the history to work out three things per alarm: when it first entered ALARM (so I know current state duration), how many state transitions it has had in the last 24 hours, and whether the most recent transition into ALARM happened overnight (roughly between 6pm the previous day and 8am today in the workspace's timezone).

Now reason about the whole list before writing anything. Group related alarms by service or dimension: same namespace, same EC2 instance, same load balancer, same RDS cluster, same Lambda function, and so on. Anything with more than three state changes in the last 24 hours goes into a 'flapping' bucket, since those usually mean a threshold or evaluation window that needs tuning rather than a live incident. Anything that transitioned into ALARM overnight goes into an 'overnight' bucket and is the most urgent. Everything else is 'ongoing'.

Post the digest to the Slack ops channel with Slack Bot's Send a Message. The top-line message text should be a single-sentence summary: total firing, how many are fresh overnight, how many are flapping, and the two or three most affected services. Underneath, list per-alarm detail grouped by service or dimension, with each alarm showing its name, current state duration, number of transitions in the last 24 hours, and its tag (overnight, flapping, or ongoing). End the message with a short 'what to look at first' section, usually the overnight fires, then anything affecting shared infrastructure, then the flappers.

Keep the digest tight. If two alarms clearly describe the same underlying incident (same host, same service, correlated timing), say that once in the summary instead of repeating detail for each. Do not paste raw AWS response fields into Slack.