Triage failed Census syncs to PagerDuty, Linear, and Slack

Every 15 minutes, sweep my Census workspace for sync runs that have failed since the last sweep and route each failure to the right escalation path across PagerDuty, Linear, and Slack.

To find failures, call Census List Syncs to enumerate the syncs in the workspace, then for each sync call Census List Sync Runs and pick out runs whose status is "failed" and whose start time falls in the window since the last sweep. Read error_code, error_message, and error_detail on each failed run so you can classify what went wrong. Census returns integer sync_id and sync_run_id values, and error fields live inside the sync run object, not the HTTP response.

Classify each failed run and take the right action:

1) Destination-API outages or transient network errors (destination timeouts, 5xx responses from the downstream SaaS, DNS or TLS problems, rate-limit blowups): page the on-call engineer through PagerDuty using Send Alert Event (Events API v2). Set severity to "error", put the sync name and a short reason in the summary, and include error_code, error_message, error_detail, and a link back to the sync in Census in the custom_details payload. Use a stable dedup_key that combines sync_id and error_code so repeated failures of the same sync with the same error group into one incident.

2) Configuration errors (missing or renamed source columns, invalid field mappings, expired or revoked destination credentials, plan or permission errors on the destination, validation errors on required fields): file a Linear Create Issue in the data-engineering team with a title like "Census sync failing: <sync name>". Put error_code, error_message, and error_detail in the description along with a link back to the sync in Census. Do not open a duplicate ticket if an open Linear issue already exists for the same sync — check first and add a comment on the existing issue instead if one is open.

3) For every failure, regardless of which of the two paths above ran, post a one-line note to the #data-ops Slack channel with Slack Send a Message: sync name, short reason (outage vs. config bug), and a link to the run in Census. This is the running log the team scans during standup.

When the failure signal is ambiguous (rare error codes, novel error text), default to filing a Linear ticket rather than paging — a false page is worse than a false ticket. If error_code is unfamiliar, use error_message and error_detail to decide, and lean on the plain-English cues: words like "timeout", "connection reset", "unavailable", "502", "503", "504" imply an outage; words like "column", "field", "mapping", "unauthorized", "invalid credentials", "scope", "schema" imply a config bug.

Dedupe so we don't spam alerts. Keep a small memory of (sync_id, error_code) pairs alerted on in the last few hours; if the same pair repeats, skip the PagerDuty page and the Linear ticket but still post the Slack log line so the team can see the sync is still failing. When a run succeeds again for a sync that was previously alerted on, clear its dedupe entry so future failures can page again.

If nothing failed since the last sweep, do nothing — no "all clear" message needed. The absence of alerts is the signal that things are healthy.