How much does a RAG chatbot cost to build and run?

A retrieval-augmented chatbot demo takes a weekend. A version you can put in front of paying customers takes considerably longer, and almost none of the difference is the model — it is the retrieval quality, the evaluation, and the work of making a wrong answer visible instead of confident.

Costs split into three parts that behave completely differently: a one-off build, a monthly run rate that scales with usage, and infrastructure. Quotes that give you one number for all three are hiding something.

The build: what you are actually paying for

Assuming your content already exists somewhere machine-readable, a production RAG system is typically a four-to-eight week build. These are the bands we scope against, in the same shape as our MVP cost breakdown.

The jump between band one and band two is almost entirely permissions and evaluation. If different users may see different documents, every retrieval must be filtered by identity before it reaches the model — get that wrong and you have built a data breach with a chat interface.

The run rate: inference

Inference bills per token, and both OpenAI and Anthropic publish current rates. Those rates fall regularly, so never build a business case on a number you read once — build it from your own token counts, which do not fall.

The variable that dominates is how much context you retrieve per question. A system that stuffs ten documents into every prompt uses roughly ten times the input tokens of one that retrieves two well-chosen passages, and it is frequently no more accurate — irrelevant context measurably degrades answers.

• Retrieve fewer, better passages. This cuts cost and improves quality at the same time, which is rare.
• Cache answers to repeated questions. Support queries cluster heavily; a cache hit costs nothing.
• Use a smaller model for the easy majority and escalate only the hard cases. Most support questions do not need your most expensive model.
• Cap per-user usage. Without a ceiling, one automated integration against your chat endpoint can produce a memorable invoice.

The run rate: retrieval and storage

You need somewhere to keep embeddings. This is a smaller cost than teams expect and a much smaller decision than vendors imply.

If you already run PostgreSQL, pgvector adds vector search to the database you are already operating, backing up and monitoring. That is the right default for most products, and it stays the right answer well past the scale at which people assume it stops. A dedicated vector database like Pinecone earns its cost at large scale or when you need features Postgres does not offer — but adopting one on day one usually buys operational overhead rather than capability.

Embedding your content is a one-off cost per document, repeated when the document changes. For most knowledge bases this is measured in single-digit pounds. Embedding models are far cheaper than generation models, so re-embedding is rarely the expensive part.

The cost everyone forgets: evaluation

The difference between a demo and a product is knowing whether it is right, and that means a test set: real questions with known-good answers, run on every change. Without it you cannot tell whether a prompt tweak helped or quietly broke something, and you will find out from a customer.

Budget for building the test set — it is a person's time collecting real questions, not an engineering task — and for the ongoing review. This is usually 15–25% of the build, and it is the line most competing quotes omit, which is part of why they look cheaper.

When RAG is the wrong tool

RAG earns its cost when answers must come from your own content and be traceable to a source. It is the wrong tool when the task is really classification, extraction or routing — those are cheaper, faster and more reliable as narrow prompts or fine-tuned models. RAG versus fine-tuning covers where the line sits.

It is also wrong when your content is bad. Retrieval faithfully surfaces whatever you have, so a chatbot over an out-of-date knowledge base produces confident, well-cited, out-of-date answers. Fix the content first; it is cheaper than any of the above.

What we charge, for reference

Most production RAG builds we take on land in the £20k–£50k band, fixed scope and fixed price, with the evaluation suite included rather than sold separately. You own the code, the prompts and the evaluation set from the first commit.

If you want the number for your specific content and volumes, that is a thirty-minute conversation and we will tell you if RAG is the wrong answer. The work itself is our RAG systems service.

Frequently asked questions

References

1. API pricing — OpenAI
2. Pricing — Anthropic
3. Embeddings guide — OpenAI
4. pgvector: open-source vector similarity search for Postgres — GitHub
5. Pinecone — Pinecone