DEVOPS

Attribute S3 storage-cost surges to the bucket, prefix, and owning repo

On a schedule, this workflow finds S3 buckets whose storage cost jumped, identifies the fastest-growing prefixes, traces each bucket to its owning GitHub repo via tags.

CategoryDevOps
Enginesim
Difficultyadvanced
Triggerschedule
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerScheduled S3 storage-cost sweep
  • ActionPull per-bucket cost deltas from DatadogDatadogDatadog
  • ActionFind fastest-growing prefixes in flagged bucketsAWS S3
  • LogicMap bucket to owning repo via tagsGitHubGitHub
  • OutputAlert owning team in Slack with attributionSlack

What it does

Narrows an S3 storage-cost surge down to the exact bucket and prefix driving it, then attributes ownership. It reads per-bucket cost and size metrics, locates the prefixes that grew most, maps each bucket to its provisioning repo from resource tags, and notifies the responsible team.

When to use it

Use it when S3 line items creep up and nobody knows which bucket or which job is writing the data. Best for teams with many shared buckets tagged with an owning repo or team.

How it works

  1. 1A schedule kicks off the storage-cost sweep on a regular cadence.
  2. 2The flow pulls per-bucket cost deltas from Datadog cloud-cost metrics.
  3. 3For each flagged bucket it inspects S3 to find the prefixes with the largest size growth.
  4. 4A logic step maps the bucket to its owning GitHub repo using the bucket's tags.
  5. 5It composes an attribution summary: bucket, top prefixes, dollar delta, owning repo.
  6. 6The summary posts to the owning team's Slack channel.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect DatadogMetrics, traces, log search.
  2. 2
    Connect AWS S3Buckets, objects, signed URLs.
  3. 3
    Connect GitHubRepos, issues, pull requests, actions.
  4. 4
    Connect SlackChannels, DMs, threads, mentions.
  5. 5
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  6. 6
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  7. 7
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.