Disclosure: This article may contain affiliate links. We earn a commission at no extra cost to you if you purchase through our links. We only recommend tools we genuinely believe in.
In April, I wrote about the support system that handled most of my tickets without me touching them. The setup was simple on purpose: a help desk, a knowledge base, an AI model with clear routing rules, and a human queue for everything uncertain. It worked because it was boring. It answered the same questions the same way, every time, and it never pretended to know something it did not know.
Six months later, the system still runs, but it does not run the way it did in April. Some parts got better. Some parts got stricter. One part I had to rebuild completely. This post is the revised playbook, not a victory lap. If you are setting support automation up for the first time, start here. If you already run a version of the April system, this is the list of changes I would make first.
It builds on How I Set Up AI-Powered Customer Support That Handles 80% of Tickets, and it borrows the lane discipline from How I Automate Demo Qualification Without a Sales Team: every ticket ends up in exactly one lane, with one clear next step.
What Has Not Changed: The Three Lanes
The core idea survived contact with six more months of real tickets. A ticket is not a score. It is a routing decision, and there are only three places it can go.
- Answer: the question is routine, the answer exists in the knowledge base, and confidence is high. Reply immediately, in your voice, with the exact steps.
- Draft: the question is plausible but the answer is not certain, the account is unusual, or money is involved in a small way. Draft the reply, hold it for one human review, send it the same day.
- Escalate: refunds and disputes, bugs, security or access problems, angry customers, and anything the model rates as low confidence. A human owns it, and the customer gets a real acknowledgement within one business day, not a silent wait.
If a ticket cannot be placed in one lane, the routing is broken, not the ticket. Fix the routing.

What Changed: The Stack Got Boring Again
In April the stack was Intercom for tickets, Notion for the knowledge base, Claude for replies, and an orchestrator in between. That shape is still right. What changed is how strict I am about each piece.
| Stage | Tool | What it does now |
|---|---|---|
| Capture | Intercom, Zendesk, or Help Scout | One inbox, one ticket per thread, no support questions answered in DMs and lost |
| Knowledge base | Notion or your help center | One source of truth, written in your voice, reviewed every month |
| Routing | n8n or the help desk workflow | Classifies into Answer, Draft, or Escalate before any reply is sent |
| Replies | Claude or ChatGPT, with your templates | Drafts from the knowledge base only, never from general knowledge alone |
| Human queue | One daily review list | Draft and Escalate tickets, cleared once a day, never as a stream of pings |
The biggest change is the rule about sources. In April, the model sometimes answered from general knowledge when the knowledge base was thin. That is how confident, wrong answers happen. Now the rule is blunt: if the answer is not in the knowledge base, the ticket goes to Draft. A missing article is a content job, not a reason to guess.
The Five Answers You Must Write Before You Automate
Do not automate a ticket type until you have written the answer yourself, in your own words, and used it at least a few times. These five cover most of the volume for a small SaaS, and they are where most teams should start.
- Password and login problems: exact steps, where the reset email lands, what to check if it does not arrive, and when to escalate for a possible security issue.
- Billing questions: renewal date, plan price, how to change payment details, and the exact words for a refund request so it routes to Escalate, not to a cheerful auto-reply.
- How to do the core thing: one feature, one path, one set of steps. If the feature has three paths, write three short answers, not one long one.
- Limits and pricing edges: what happens at the plan limit, what an upgrade changes, and what it does not change. This is where vague answers cost trust.
- Cancellation and downgrade: offer the downgrade or pause first, in plain language, and process the cancellation promptly if the customer still wants it. A retention attempt is fine. A retention trap is not.
Write each answer the way you would actually say it. Short sentences, contractions, the customer name where you have it, and no corporate filler. The model will copy your tone. Give it a good tone to copy.
The Confidence Rule, Written Down
Routing by feel is how a system slowly drifts. Write the rule down, in the workflow itself, so a new tool or a new team member cannot quietly change it.
| Signal | Lane | Why |
|---|---|---|
| Answer exists in the knowledge base, wording matches the question | Answer | Low risk, high repetition, safe to send |
| Answer exists, but the account or wording is unusual | Draft | A human check costs minutes and prevents a wrong reply to the wrong person |
| Money, refunds, disputes, or account access is involved | Escalate | The cost of a wrong answer is higher than the cost of waiting a day |
| No matching article, or the model is unsure | Draft, then write the article | Uncertainty creates content, not guesses |
| Customer is angry, or the same issue repeats | Escalate | Tone and repetition are signals, not noise |
One more rule from April that I now enforce harder: never auto-reply to a cancellation, refund, or complaint with a template that sounds like a celebration. Read it out loud before it ships. If it sounds wrong to a frustrated person, it is wrong.
The Monthly Review That Keeps It Honest
Automation rots quietly, exactly like the blog did when its one pipeline died. The fix is a short monthly review, budgeted as a planning figure, not a measured average: about 45 minutes, once a month.
- Read five Answer replies at random. Is the answer still correct, and does the linked article still exist?
- Clear the Draft queue and count what landed there. If one question keeps landing in Draft, write the article it needs.
- Check the Escalate queue for patterns. Three tickets about the same bug are a product note, not three support tickets.
- Delete or rewrite one stale article. Old answers in an automated reply are still your answers.
- Test the whole path once: send a test ticket, check the reply, check the links. Automation that silently fails is worse than no automation, because you stop checking.
This review is also where the 80% idea gets honest. The goal is not to push the auto-answer rate higher every month. The goal is to keep the right tickets automated and the wrong ones human. A system that answers 80% of tickets well beats a system that answers 95% of tickets badly, and the monthly review is how you tell which one you have.
What I Rebuilt: The Handoff Note
The one part I rebuilt completely was the handoff from AI to human. In April, an escalated ticket arrived with the original message and not much else, so I re-read the whole thread every time. Now every Draft and Escalate ticket carries a short handoff note, written by the model before a human touches it.
- What the customer asked, in one sentence.
- What answer was considered, and why it was not sent.
- What is missing: an article, an account detail, a decision only a human can make.
- The suggested next step, if there is an obvious one.
That note saves the re-read, and it makes the human queue faster to clear. It also makes the system teachable: when the same missing detail shows up three times, you know exactly what to fix.
Lessons at the End
- Lanes beat percentages. An auto-answer rate is trivia. A lane with a clear next step is a system.
- The knowledge base is the product. The model is interchangeable. The articles, in your voice, are not.
- Uncertainty is a lane, not an error. Draft exists so the system never has to guess on a ticket that matters.
- Write the answer yourself first. If you have never answered the question well as a human, the model will not invent a good answer for you.
- Review monthly, or it rots. A support system nobody checks will eventually answer confidently, wrongly, and at scale.
- Keep a human queue, and clear it daily. The queue is not a failure of automation. It is the part that keeps automation trustworthy.
If you build only one thing from this post, build the three lanes and the confidence rule, and write five good answers before you turn anything on. Run it for a month, read the Draft queue, and let that queue teach you what to write next. That is the whole playbook, revised.
