Deploy maintenance window console for Grafana alerts

By General Input

Open a maintenance window before every deploy, silence exactly the right Grafana alerts, and get them back on automatically.

Integrations

  • Grafana
  • Linear

Type

App

Categories

  • Engineering
  • Operations

Build me an app my team opens before every deploy or planned maintenance, so we silence exactly the right Grafana alerts instead of muting everything by hand and forgetting to unmute. The people using it are engineers about to ship, plus whoever is on call and wants to know what is currently quiet.

The main view is a single console with three panels side by side. The first panel is what is currently suppressed, read from Grafana with List Mute Timings, split into windows active right now and windows scheduled to start later, each showing its time range, when it ends, the service it covers, and who opened it. The second panel is the alert rules from List Alert Rules grouped by folder, with each rule marked quiet or live depending on whether a currently active window covers it. The third panel is the notification targets from List Contact Points. Tie the three together so each rule answers the question 'where would this page if it fired right now': read the routing tree with Get Notification Policy Tree and match each rule's labels against the routes to resolve the contact point it would land on, then show that contact point next to the rule. Let me filter the whole view by folder, by service, and by quiet-versus-live.

To open a window I pick a service or a folder, set a time range, and type a short reason. On confirm the app suppresses the alerts with Create Mute Timing using a predictable name that encodes the service and the date so windows are easy to spot and delete later, drops a marker on each affected dashboard with Create Annotation covering the same time range and carrying the reason plus a shared tag so the app can find its own markers later, and files a change record in Linear with Create Issue containing the service, the time range, the reason, who opened it, and the list of rules that went quiet. Store the window record in the app, including the mute timing name, the annotation ids, and the Linear issue url, so the console can link everything back together.

One thing to get right: in Grafana a mute timing only takes effect if a notification policy route actually references it. After creating a window, check the routing tree with Get Notification Policy Tree and show me plainly whether the window is really in effect or whether it was created but is not attached to any route. If it is not attached, surface that as a warning in the console rather than pretending the alerts are quiet, and offer an explicit confirm action that attaches the new window to the routes I select, leaving the rest of the tree untouched.

Each active window has an End early button that removes it with Delete Mute Timing and updates the record so the history shows the real end time rather than the planned one. A history view lists past windows using Find Annotations filtered to the app's tag, joined with the stored records, showing service, planned versus actual duration, reason, who opened it, and a link to the Linear change record. Let me search that history by service and by date range.

Add a Prepare window button that runs a background agent. It takes the service name I typed and works out the scope for me: it finds the matching dashboards and folders with Search Dashboards and Folders, pulls the rules with List Alert Rules, and resolves where those rules would page using Get Notification Policy Tree and List Contact Points. It writes its proposal back into the app as a pre-filled window: the folders and rules it wants to suppress, the dashboards it will annotate, a suggested time range, and a draft reason. Critically, it also writes a short note listing anything it deliberately left live and why, for example customer-facing availability rules or alerts that belong to a different service that happens to share a name. The agent never opens the window itself. I review the proposal, edit the selection, and confirm.

A few rules to bake in. Windows always carry a fixed end time, with a configurable default length and a hard maximum so nobody can accidentally silence something for a week. Record the person who opened each window and show a 'my windows' filter alongside the full list. Never suppress anything without an explicit confirm, and always show the exact list of rules that will go quiet before I confirm. If the reason field is empty, block the window, since the whole point is that the annotation and the change record explain the gap later.

Related prompts

Explore more prompts
Catch feature flags that never got switched on in productionWeekly alert noise review board for incident.io on-callMorning triage board for your Kernel browser agent runsTrack every half finished LaunchDarkly rollout in one boardFind the customers stuck contacting support again and againTurn shipped Linear work into Intercom product announcementsVoice agent QA review board for your Hume EVI callsClose the loop between Front tickets and Linear bug fixesOpen a Heroku release war room the moment production looks wrongFind the themes Help Scout tags miss and file them in Linear