Automation first, AI second: use LLMs only where judgment is needed
TL;DR
Steps with clear rules (filtering, format conversion, API calls, math) → plain logic: cheap, fast, stable, testable. Steps needing language understanding, fuzzy classification, summarizing, drafting → an LLM. Result: lower cost and fewer unpredictable failures.
With an LLM at hand it’s tempting to “let AI do everything”: AI reads email, AI filters spam, AI calculates refunds, AI decides who gets what. The result is usually an expensive, slow system that occasionally fails in unpredictable ways. The opposite principle works far better: automate with ordinary logic first, and use the LLM only for steps that genuinely need judgement.
Two kinds of steps in every process
| Deterministic steps (clear rules) | Judgement steps | |
|---|---|---|
| Examples | Filter by sender, reformat dates, call an API, calculate, match IDs | Understand intent in an email, fuzzy classification, summarise, draft a reply |
| Tool | if/else, regex, SQL, code, built-in n8n nodes | LLM |
| Cost | Practically zero | Tokens on every run |
| Speed | Milliseconds | Seconds |
| Reliability | 100% repeatable and testable | Probabilistic; needs evals |
Example: handling refund-request emails
| Step | Use | Why |
|---|---|---|
| Skip auto-replies, newsletters, spam | Rules | Headers and sender are enough; no tokens spent |
| Read the content and work out what the customer wants | LLM | Natural language, many ways of saying it |
| Extract the order ID | Regex first, LLM as fallback | IDs have a fixed format; only ask the LLM when regex fails |
| Look up the order in the database | Code / API | Structured data |
| Calculate the refund, check the return window | Code | Money can’t be “approximately right” |
| Draft a polite reply | LLM | Needs context-appropriate wording |
| Approve refunds above a threshold | Human | Financial risk |
Of seven steps, only two or three need an LLM. The rest are cheaper, faster and testable with ordinary unit tests.
Why this works better
- Cost: every step moved from LLM to code removes one model call per record.
- Reliability: LLM failures are hard to predict; code failures come with stack traces and can be reproduced.
- Debuggability: when output is wrong, you know which step to look at.
- Swappability: a small, well-defined LLM part means changing models or prompts doesn’t ripple through everything.
It’s also the spirit of Anthropic’s “Building effective agents”: find the simplest solution possible and add complexity only when you can measure that it improves results. Many problems need nothing more than one well-designed LLM call with retrieval and examples — no agent required.
Four questions for every step
- Can I write if/else, a regex or SQL for it? Yes → don’t use AI.
- How costly is one mistake? High (money, legal, key customers) → human approval; AI only proposes.
- Does the output need a fixed structure? Yes → enforce a JSON schema (structured outputs) and validate.
- Do I have sample data to evaluate it? Not yet → prepare 50–100 hand-labelled examples before going live.
The “sandwich” architecture
[Code: filter and normalise input]
↓
[LLM: understand / classify / draft] → JSON per schema
↓
[Code: validate, look up, calculate, route]
↓
[Human: approve when risk is high / confidence is low]The LLM is “sandwiched” between two layers of code: the first trims and cleans the input, the second checks and executes. LLM output never flows straight into an action with consequences.
How to apply it
- Developers: Review existing workflows: which LLM steps could be rules? Start with input filtering — it usually saves the most tokens.
- Tech Leads: In design reviews, ask for workflow diagrams that clearly colour the “deterministic part” and the “AI part”.
- PMs: When estimating automation projects, split the two. The AI part needs extra time for evaluation data, prompt tuning and edge cases — don’t estimate it like ordinary code.
Sources & references
- Building effective agents — Anthropic (Erik Schluntz, Barry Zhang)
- Structured outputs — Claude Docs