Chatbot vs RAG vs AI Agent
The capability ladder for enterprise AI. Start lower than you think — most "AI agent" projects should be RAG.
TL;DR: Chatbot = scripted FAQ (cheap, fast, limited). RAG = chatbot + your documents (most enterprise needs stop here). Agent = RAG + tools + reasoning loop (powerful but failure-prone). Match capability to problem; don't pay for what you won't use. ~70% of "we need an AI agent" projects we audit should be RAG.
Level 1
💬 Chatbot (scripted / FAQ-style)
Decision-tree or shallow LLM bot. Handles known questions with known answers. No retrieval, no tools.
Cost: $20-80K
Build time: 3-6 weeks
Failure mode: Falls off-script → frustration
Use when:
- You have a small, stable FAQ (~50-200 entries)
- Customer expectations are narrow (e.g. "what time do you open?", "can I cancel?")
- Your goal is deflection of repetitive calls, not deep problem-solving
- Your data fits in the system prompt (no need for retrieval)
Don't use when: Knowledge changes weekly, you have hundreds of documents, or you need actions taken on behalf of the user.
Level 2
🔍 RAG (Retrieval-Augmented Generation)
Chatbot + retrieval over your documents. LLM answers grounded in your knowledge base, with citations. Read-only.
Cost: $60-200K
Build time: 6-12 weeks
Failure mode: Bad retrieval → confident wrong answers
Use when:
- You have a body of documents (policies, products, contracts, manuals)
- Knowledge updates regularly (RAG re-indexes; fine-tune doesn't)
- Users ask "what does X say about Y?" or "where is Z documented?"
- You need source citation for compliance/audit
- This is the right answer for ~70% of enterprise "AI" projects
Don't use when: You need actions executed (booking, cancellation, write operations) — that's Level 3.
Level 3
🤖 AI Agent (RAG + tools + reasoning loop)
RAG bot that can call tools, take actions, and iterate. Books appointments, cancels orders, queries databases, fires off workflows.
Cost: $150-450K
Build time: 12-24 weeks
Failure mode: Premature execution → real-world consequences
Use when:
- The user wants something done, not just answered
- Multi-step workflows where the agent needs to decide which tools, in which order
- You have stable APIs the agent can call (write operations included)
- You can afford the failure mode: agents take wrong actions ~5-15% of the time even with safeguards
- Confirmation patterns + idempotency are designed in from day 1
Don't use when: Stakes of a wrong action are high (financial transactions over $X, irreversible operations) without strong confirmation gating. Or when the user's task is just informational — agents are overkill for that.
The Upgrade Path
Start at the lowest level that solves the problem. Each upgrade is ~3-5x cost and complexity:
- Chatbot → RAG: Common when the FAQ explodes past 200 entries or knowledge starts updating weekly. Migration: 4-8 weeks. Worth it when 30%+ of bot answers are "I don't know" or "let me get someone".
- RAG → Agent: Common when users are asking the bot to "do X" not "tell me about X". Migration: 3-6 months. Worth it when the tool surface area is well-defined and the failure cost is manageable.
- Skip-levels: Rare and dangerous. If you skip RAG to build an agent without grounded knowledge, you get an agent that confidently does wrong things based on hallucinations.