Lorixen.
EN

RAG or fine-tuning for a business AI assistant?

Short answer: For a business assistant that has to answer from your own policies, prices, products and documents, start with retrieval (RAG): the assistant looks up the relevant passages at question time and answers from them, so updating a document updates the answers. Fine-tuning changes how a model behaves, such as tone, format or a narrow classification task, and it's a poor way to load facts that change. Most small-business assistants need good retrieval, clear instructions and a test set; fine-tuning comes later, if at all, and availability varies by provider.

Founders hear both terms in the first meeting about an AI assistant, usually as if they were competing products. They aren't. They solve different problems, and mixing them up leads to the most common mistake: paying to "train the AI on our data" when what you needed was for it to look things up.

The plain-English version

Retrieval-augmented generation (RAG). Your documents (FAQs, policy pages, product sheets, manuals) are split into passages and indexed. When a customer asks a question, the system searches that index, pulls the most relevant passages, and gives them to the model along with the question. The model writes an answer from what it was handed. OpenAI's file search tool is one hosted example: you upload files to vector stores, and the model can retrieve from that knowledge base using semantic and keyword search. Anthropic's embeddings documentation covers the building block for doing retrieval yourself and points to embedding models from a third-party provider, including models it describes as optimized for retrieval and RAG.

Fine-tuning. You train a base model further on many examples of inputs and the outputs you want. OpenAI's model optimization guide describes it as getting a model that excels at the tasks you'll use it for, and lists benefits such as consistent formatting, shorter prompts at scale, and training a smaller, cheaper model for a specific task. Its supervised fine-tuning examples include classification, nuanced translation, content in a specific format and correcting instruction-following failures.

Notice what isn't on that list: "know our current return policy".

Side by side

Retrieval (RAG) Fine-tuning
What it changes What the model can see when answering How the model tends to respond
Best for Facts, policies, prices, product details, documents Tone, output format, classification, narrow repetitive tasks
Updating information Edit or replace the document; re-index Prepare new examples and retrain
Showing sources Can quote or link the passage it used Can't point to where an answer came from
Data you need Your existing documents, cleaned up Many good input/output examples
Main risk Poor search results lead to poor answers Confidently outdated or invented facts
Provider availability Widely available Varies; check each provider's current docs

That last row is worth a closer look. As of November 2026, OpenAI's guide says it is winding down its fine-tuning platform: it's no longer accessible to new users, existing users can create training jobs for the coming months, and fine-tuned models remain available until their base models are deprecated. Other providers have their own offerings and terms. A plan that depends on fine-tuning needs a provider check first.

When retrieval fits

  • A customer service assistant answering from your help center and policies.
  • A staff assistant that searches procedures, product specs or past proposals.
  • A phone or chat assistant quoting opening hours, service areas or price ranges.
  • Anything where information changes monthly, weekly or daily.

Retrieval quality depends far more on your content than on the model. Clear, current, non-contradictory documents produce good answers; three versions of the same policy produce confused ones.

When fine-tuning might fit

  • Sorting incoming messages into categories at high volume, where a smaller tuned model is cheaper per message.
  • Producing output in a strict format your systems need, every time.
  • A distinctive house style that prompting and examples can't hold consistently.

Even then, OpenAI's guide places fine-tuning inside a loop: write evals, prompt the model with good context and examples, fine-tune only for some use cases, then measure and repeat. It also notes that prompt engineering may be all you need.

The hybrid most businesses end up with

In practice, a solid business assistant usually combines:

  1. Instructions (a system prompt) for tone, boundaries and what to do when unsure.
  2. Retrieval for facts, with sources.
  3. A few examples in the prompt for tricky formats.
  4. Tools for live data, such as checking an order status or booking availability, rather than storing that data in documents.
  5. Fine-tuning, only if tests show a specific gap that the first four can't close.

Questions to ask whoever builds it

  • Which documents will it search, who keeps them current, and how quickly do edits show up?
  • Can it show or link the source behind an answer?
  • What happens when nothing relevant is found? (The right answer is "it says so and offers a person".)
  • How will you test it, and can we see the test results?
  • What are the running costs per conversation, and what drives them up?
  • If you propose fine-tuning: why, on which provider, with what training data, and what happens if that provider changes its offering?

Build a test set before you build anything

The single most useful thing you can do is write 30 to 50 real questions customers ask, with the correct answers, including awkward ones: edge cases, out-of-date policies, questions you don't want answered. OpenAI suggests even writing evals before writing prompts. Run every version of the assistant against this set. It turns "it seems good" into something you can compare, and it's how you'll know whether a change (new documents, a different model, fine-tuning) actually helped.

Verdict

  • Answering from your business information: retrieval, with good documents and source links.
  • Consistent format or high-volume classification: consider fine-tuning after prompting and evals show a real gap, and only on a provider that offers it.
  • Live data (orders, bookings, stock): connect tools to your systems instead of either approach.
  • Everyone: a test set first.

FAQ

Can't I just fine-tune the model on our website?

You could try, but facts learned through training are hard to update and can't be traced back to a source. Retrieval keeps answers tied to your current content.

Does RAG stop the assistant making things up?

It reduces the risk by giving the model the right material, but it doesn't remove it. Instructions to answer only from sources, source links, and testing all help.

Is RAG expensive?

It adds search and indexing to each question, and longer prompts cost more. Ask your builder for a per-conversation estimate based on your expected volume.

How often should documents be updated?

Whenever the underlying information changes. Assign an owner for each document set so the assistant doesn't quote last season's prices.

When you're scoping an assistant that has to know your business, we can map out the retrieval, tools and tests it needs.

Sources

WhatsAppLet’s talk