AI triage for email and tickets: route to the right person in seconds

Every morning the support inbox holds dozens to hundreds of new emails and tickets, and someone has to read each one to decide: bug or question, urgent or not, who should take it. It’s repetitive, time-consuming and slows customer response — exactly the kind of work LLMs do well. This article lays out a complete AI triage flow: filter, classify to a schema, route, and measure so you can trust the system.

The flow

1. Trigger: a new email (Gmail/Outlook) or ticket (Jira Service Management, Zendesk, Freshdesk) via webhook or polling.
2. Rule-based pre-filter: spam, auto-replies, newsletters and system notifications → skipped with simple conditions (no tokens spent).
3. LLM classification: reads subject + body and returns JSON in a fixed schema.
4. Code-based routing: apply labels, assign owners, set SLAs by priority — all deterministic logic.
5. Low confidence: below a threshold → a “needs human review” queue.
6. Logging: store input, output and the final decision for evaluation and prompt improvement.

Suggested schema

{
  "category": "bug | billing | how_to | feature_request | other",
  "priority": "P1 | P2 | P3",
  "product_area": "login | payment | report | api | other",
  "language": "vi | en",
  "summary": "one-sentence summary of the issue",
  "customer_sentiment": "negative | neutral | positive",
  "confidence": 0.0
}

Use structured outputs (JSON schema) so the model always returns the right shape. The code after it should still validate: any value outside the allowed list → treat as low confidence.

Classification prompt

You classify support tickets for a point-of-sale software product.
Priority definitions:
- P1: system unusable, data loss, wrong charges, many users affected.
- P2: one feature broken but a workaround exists.
- P3: how-to questions, feedback, feature requests.
If unsure, set a low confidence rather than guessing.


... 10–20 real, labelled tickets ...


Ticket to classify:
{{subject}}\n{{body}}

Clearly defining each priority and including real examples (few-shot) usually improves accuracy more than switching to a bigger model.

Measure before you trust

For P1, a miss is far worse than a false alarm. Add keyword rules (“can’t log in”, “data lost”, “charged twice”) that raise priority regardless of the AI’s classification.

Roll out in stages

1. Shadow mode (1–2 weeks): the AI classifies but only logs; people work as before. Compare to measure accuracy.
2. Suggestions: the AI applies a “suggested” label; a human confirms with one click.
3. Thresholded automation: high confidence → auto-route; low → human queue.
4. Expand: the AI drafts replies for how-to tickets; agents edit and send.

Common pitfalls

• Too many categories: 30 fine-grained labels get confused; start with 5–7 groups.
• Personal data: mask card numbers and phone numbers before sending to the model if your policy requires it.
• Prompt injection: email content is untrusted data — the AI may only classify, never act on “instructions” inside an email.

How to apply it

• Developers: Build it in n8n or a few dozen lines of code; keep the AI part a pure “text → JSON” function so it’s easy to test.
• Tech Leads: Build a 100–200 ticket labelled eval set; rerun it whenever the prompt or model changes.
• PMs / Support Leads: Agree on written P1/P2/P3 definitions first — the AI can’t be more consistent than the team’s own definitions.