Weekly refresh of staging with anonymized production data

By General Input

Every Sunday night, rebuild your shared staging database from a masked copy of production so your team tests on realistic data, never real customer records.

Integrations

  • Neon
  • Slack

Type

Agentic Task

Categories

  • Engineering
  • Operations

Every Sunday at 11pm, refresh our shared staging database in Neon with a fresh anonymized copy of production, so the team always tests against realistic data without ever handling real customer records.

Start by creating the new staging database. Use Neon's Create anonymized branch operation to branch our production branch with masking rules applied, so customer names, emails, and other personal fields are replaced with realistic fake values. Name the new branch with a clear dated convention such as staging-YYYY-MM-DD, so it is obvious when it was created and which branch it replaces.

Anonymization is a multi-step process. The create call only starts the work, it does not finish it, and a successful create response does not mean the data has been masked yet. After creating the branch, poll Retrieve anonymized branch status until anonymization actually reports as complete, then confirm the branch itself is in a ready and healthy state before going any further. Never treat the create response alone as a finished refresh.

Only once the new branch is confirmed anonymized and healthy should you remove the previous week's staging branch. Use List branches to find the prior staging branch by our naming convention, then Delete branch to remove it. The ordering matters: always create and verify first, then delete, never the reverse, so there is never a moment where the team has no staging database at all. Only ever delete branches that match our staging naming pattern, and never delete the production branch or the project's default branch. If Neon refuses the delete because the branch still has child branches or running operations, leave it in place and mention that in the Slack message rather than forcing it.

When the refresh lands successfully, post a Slack message to our engineering channel with the new branch name, how long the whole refresh took from start to finish, and a link to the Neon console for the project. Never include the database connection string, password, or any live credentials in that message, and never write them into logs or saved output either. That channel is shared with people who should not have production-shaped database credentials, so the console link is the only access path we ever put in chat.

If anonymization fails, or if it runs much longer than usual, stop and do not delete anything. Treat roughly 45 minutes as the ceiling for a normal refresh and make that threshold easy to adjust, since it scales with database size. In that case leave the previous staging branch completely untouched, so the team is never left without a working staging database, and post a Slack warning instead. The warning should say that the refresh did not complete, which stage it failed at, how long it ran before giving up, and that last week's staging branch is still live and safe to keep using.

Neon's anonymization capability is currently in beta. If the operation is unavailable on our plan or returns an unsupported or not-permitted error, handle it exactly like a failure: keep the existing staging branch, delete nothing, and post the Slack warning so we know to look at it.

Related prompts

Explore more prompts
Call overdue Xero customers with an AI collections agentWin back LiveChat visitors whose chats went unansweredChat quality review board for LiveChat support leadsWin back no-show and cancelled appointments every morningLive Loop returns analytics with product-level drill-downNewsletter pre-flight and approval board for Mailjet sendsTurn a prospect spreadsheet into personalized sequence enrollmentsMailjet email delivery lookup console for support teamsCatch feature flags that never got switched on in productionKajabi customer support console for member access fixes