Weekly Service Baseline Drift Digest

Every week this workflow reviews a watchlist of service metrics, pulls each one's recent Datadog trend against its longer-term baseline, and identifies metrics whose normal range has quietly shifted. It cross-references the documented thresholds in Confluence runbooks, then posts a single Slack digest highlighting drift so the team can update alert thresholds and runbook expectations before the drift becomes an incident.

Use it to keep alerting honest. Baselines decay as traffic patterns and deploys change; this catches the slow drift that makes monitors either too noisy or dangerously quiet.

1. 1A weekly schedule trigger starts the run.
2. 2Datadog returns recent-week and trailing-90-day series for every metric on the watchlist.
3. 3A logic step flags metrics whose recent baseline diverges materially from the long-term one.
4. 4For each flagged metric, Confluence is searched for the runbook's documented threshold.
5. 5OpenAI writes a digest noting drift direction, magnitude, and whether the runbook threshold is now stale.
6. 6The digest posts to the team's Slack channel with per-metric runbook citations.