Flag risky Fivetran pipelines every Monday in Slack
Every Monday, we review how your Fivetran pipelines handle new data and post a plain-English Slack summary that flags the ones quietly driving up cost.
Integrations
Fivetran
Slack Bot
Type
Categories
- Operations
- Engineering
Every Monday at 9am, audit how my Fivetran connections handle schema changes and post a data governance summary to a Slack channel so my data platform owner can review risky settings each week. This is a point-in-time posture review of the current configuration, not a comparison over time, so no baseline or history is needed.
Start by calling Fivetran's List Connections operation to retrieve every connection (connector) in the account.
For each connection, call Fivetran's Get Connection Schema Config operation to retrieve its full schema, table, and column configuration, including the schema_change_handling setting (typically ALLOW_ALL, ALLOW_COLUMNS, or BLOCK_ALL).
Flag any connection that is configured to automatically allow all new schemas, tables, or columns, meaning schema_change_handling is set to ALLOW_ALL, because those pipelines silently ingest new source data and can inflate monthly active rows and cost without anyone approving the change. Treat ALLOW_COLUMNS as a middle posture where new columns are added automatically but new tables and schemas are held, and treat BLOCK_ALL as the safest posture where nothing new is ingested until it is approved.
Group the connections into two buckets: safe versus permissive. Put the ALLOW_ALL connections in the permissive bucket and call out which ones have the broadest auto-allow posture first, so the reviewer can act on the riskiest pipelines. Put the more restrictive connections in the safe bucket.
Write a clear, prioritized summary and send it to a Slack channel using Slack Bot's Send a Message operation. Lead with the permissive connections that automatically pull in new data, name each one, and briefly explain why it is risky in cost terms. Then list the safe connections. Keep the wording plain by saying 'automatically pulls in any new data' rather than the internal setting name, so a non-technical reviewer can act on it. Post to a data governance channel such as #data-governance, and make the channel easy to change.
What it does
- Checks every one of your Fivetran connections each week to see how it handles new source data.
- Flags the pipelines set to automatically pull in any new tables, columns, or data sources, since those can quietly grow your data volume and cost.
- Sorts your connections into safe versus permissive so you can see the riskiest ones first.
- Posts a plain-English governance summary to a Slack channel your data owner can review in minutes.
What you’ll need
- A Fivetran account with the connections you want to review.
- A Slack workspace and a channel where the weekly summary should be posted.
How to customize it
- Change the day or time, for example run it every Friday afternoon instead of Monday morning.
- Pick which Slack channel gets the summary, like a shared #data-governance channel or a private review channel.
- Adjust what counts as risky, for example treat pipelines that auto-add new columns as safe or as worth flagging.
Use cases
- AI Reports
- Notifications & Alerts
