CUSTOMER SUPPORT

Macro Effectiveness Loop: Track Which Canned Replies Resolve Threads and Retire the Duds

Monthly, measures how often each Intercom saved reply leads to a quick resolution versus a reopened or escalated thread.

CategoryCustomer Support
Enginesim
Difficultyintermediate
Triggerschedule
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerMonthly schedule fires
  • ActionPull saved-reply usage and outcomes from IntercomIntercomIntercom
  • LogicAggregate usage, resolution, reopen, and escalation per macro
  • LogicScore effectiveness and flag underperformers
  • OutputWrite the ranked effectiveness report to NotionNotionNotion

What it does

This workflow closes the loop on macros you've already shipped. Each month it looks at every saved reply, tallies how many conversations used it, and measures what happened next: did the thread close quickly, get reopened, or escalate to a human rewrite. It scores each macro's effectiveness, writes a ranked report to Notion, and flags the underperformers that confuse customers or fail to resolve so your team knows what to fix or retire.

When to use it

Use it after you've built a macro library and want evidence about which canned replies actually work, instead of letting stale or misleading ones quietly pile up.

How it works

  1. 1A monthly schedule triggers the analysis.
  2. 2Pull conversations from Intercom that used a saved reply in the last 30 days, with their outcomes.
  3. 3Aggregate per macro: usage count, fast-resolution rate, reopen rate, and escalation rate.
  4. 4Compute an effectiveness score and rank every macro.
  5. 5Flag macros below the effectiveness floor as candidates for rewrite or retirement.
  6. 6Write the ranked report and flagged list to a Notion page for the team.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect IntercomConversations, contacts, articles.
  2. 2
    Connect NotionPages, databases, comments.
  3. 3
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  4. 4
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  5. 5
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.