Catch feature flags that never got switched on in production

By General Input

Every weekday we compare your staging and production feature settings and flag only the differences that could be hurting customers.

Integrations

  • LaunchDarkly
  • Slack
  • Linear

Type

Agentic Task

Categories

  • Engineering
  • Operations

Every weekday at 8am, audit my LaunchDarkly feature flags for configuration drift between environments and tell me which mismatches are actually risky. Compare my staging environment against my production environment. Keep the LaunchDarkly project key, the two environment keys, the Slack release channel, and the Linear team as configuration at the top of the workflow so they are easy to change later.

Start by pulling every flag in the project with its per environment configuration using LaunchDarkly's "List feature flags" operation, following pagination until you have the full set. Then call "List feature flag statuses" once for staging and once for production, so you know which flags are actively being evaluated in each environment and when each one was last requested. For any flag whose targeting looks like it diverges between the two environments, call "Get feature flag" to inspect the per environment configuration closely before you judge it.

For each flag, compare staging against production on three things: the on/off state, the default variation served when the flag is on and when it is off, and the targeting rules, including percentage rollouts and individually targeted contexts.

Then judge each difference instead of reporting all of them. Differences that are usually intentional include a flag mid rollout where production is serving a percentage rollout, an environment only kill switch, and a flag that is brand new and has only been configured in staging so far. Differences that are usually dangerous include a flag that is on and actively evaluated in staging but has never been enabled in production, which normally means a finished feature customers never received, and the two environments serving different variations with no rollout in progress. Treat a flag that is off in staging but on in production as risky too, since it means something is live for customers that nobody is exercising in a test environment. Use the last requested timestamps to tell a genuinely live flag apart from one that no code path touches any more.

Post one ranked digest to my release channel with Slack's "Send a Message" operation, ordered by risk with the most dangerous mismatches first. List only the risky mismatches, and for each one give the flag key, exactly what differs between staging and production, and which environment looks wrong. Write it so an on call engineer who has not opened LaunchDarkly yet can act on it, and quote flag keys directly, since LaunchDarkly addresses flags by human readable project, environment, and flag keys rather than ids. On days when nothing is worth flagging, stay silent and post nothing at all.

For anything ranked high risk, open a Linear issue with the "Create Issue" operation, naming the flag key in the title and describing the drift and which environment looks wrong in the body, so it gets an owner. Before creating one, use "Search Issues" to look for a matching issue for that same flag key created in the last two weeks, and skip it if one already exists so we never file duplicates.

This must stay strictly read only against LaunchDarkly. Report and ticket the drift, and never toggle a flag, edit targeting, or copy flag settings between environments automatically. Do not build any part of this on LaunchDarkly approvals, scheduled changes, flag triggers, or the copy feature flag operation, since those are plan gated and fail on lower plans. Remediation stays manual by design.

Related prompts

Explore more prompts
Call overdue Xero customers with an AI collections agentWin back LiveChat visitors whose chats went unansweredChat quality review board for LiveChat support leadsWin back no-show and cancelled appointments every morningLive Loop returns analytics with product-level drill-downNewsletter pre-flight and approval board for Mailjet sendsTurn a prospect spreadsheet into personalized sequence enrollmentsMailjet email delivery lookup console for support teamsKajabi customer support console for member access fixesRun your application review round on Jotform submissions