Turn a wall of CloudWatch alarms into a filed postmortem

I want an incident review workbench I open the morning after something breaks, so my team can turn a messy pile of CloudWatch alarm noise into a real postmortem without anyone scrolling the AWS console at 2am. This is an app a person sits in and works through, not a scheduled report. Nothing in it should run on a timer or a cron schedule.

The app opens on an incident setup view where I pick a start time and an end time for the incident window. When I confirm the window, the app pulls my alarms with Describe Alarms and then calls Describe Alarm History for each alarm, filtered to that date range and to state transition items, and merges everything into a single chronological timeline. Every entry shows the alarm name, the timestamp, the direction of the change such as OK to ALARM or ALARM to OK, and the reason CloudWatch recorded for it. Alarms whose first transition into ALARM happened earliest are pinned to the top of the timeline and labelled as first movers, because the first thing to fire is usually the closest thing we have to a root cause.

Clicking any alarm on the timeline opens a detail panel. It shows that alarm's configuration from Describe Alarms, including the metric it watches, the threshold, the evaluation periods, and the namespace and dimensions. For an alarm currently in ALARM state it also calls Describe Alarm Contributors to show which individual time series inside that alarm were actually in breach, so I can tell whether one instance broke or the whole fleet did. The panel has a button to chart the underlying metric across the incident window using Get Metric Statistics, so I can see the shape of the failure, whether it was a cliff, a ramp, or a sawtooth, without leaving the app.

The curation step is the heart of this app and must not be skipped or automated away. Next to every alarm on the timeline I can tag it as part of the incident, collateral damage, or unrelated noise. Only alarms I tagged as part of the incident or as collateral feed the postmortem. This human separation of real signal from noise is the entire point of the workbench, so make it fast and obvious, with quick keyboard friendly ticking and a running count of how many alarms are still untriaged.

Once I have curated the timeline, the app drafts a structured postmortem. It should include the detection time, the alarm that caught the incident first, the time to first response, the total duration, and the affected resources gathered from the dimensions on the alarms I kept. Be honest in the wording about where these numbers come from. Detection and response times are inferred from when alarms changed state, not from any paging or acknowledgement tool, so label them that way in the draft, for example 'first response inferred from the gap between the first ALARM transition and the first recovery activity'. Never imply we have real acknowledgement tracking. Every field in the draft is editable inline before anything leaves the app.

When the draft looks right, a File to wiki button creates the postmortem as a new page in our engineering wiki using the Notion Create a Page handler, under a parent page or database I choose in settings. The app stores the resulting Notion page URL against the review.

In the same workbench I add follow up actions, each with a title, an owner, and a short description. Filing them creates one ticket per action through the Linear Create Issue handler, into a team I pick, and every ticket description links back to the filed Notion postmortem page so the context travels with the work. Once filed, the actions stay listed in the app with their Linear ticket links, and I can edit a ticket's title or description from here using Linear Update Issue rather than switching tools.

The app keeps a list of every past review it has produced, stored in the app itself, showing the incident window, the first mover alarm, the duration, the postmortem link, and the follow up tickets. Across that history the app counts how many separate incident reviews each alarm has appeared in, and flags any alarm appearing in three or more as a chronic offender, because that is a problem nobody has actually fixed. Surface these chronic offenders prominently on the past reviews view, not buried in a submenu.

Two scope limits to build around. CloudWatch Logs is a separate service and is not part of this integration, so the postmortem cannot contain log excerpts or error traces. Keep the evidence strictly to alarm state transitions and metric shapes, and do not write UI copy that promises log analysis. Also, CloudWatch only retains alarm history for a limited period, so handle an empty or truncated history window gracefully. If I pick a window with no history or only partial history, say plainly that the history is unavailable or incomplete for that range rather than rendering an empty timeline or implying the incident had no alarms.

Read only CloudWatch access is enough for everything on the AWS side. The app never creates, edits, deletes, or silences an alarm.