AI··1 min read
Two ways to make a language model know about your business. They solve different problems, and picking the wrong one is expensive.
When teams want an LLM that "knows our stuff", they usually hear two options: retrieval-augmented generation (RAG) or fine-tuning. They get talked about as if they were competitors. They are not. They fix different things.
RAG changes what the model can see
With RAG, you keep your documents in a searchable store. When a question comes in, the system finds the most relevant passages and hands them to the model along with the question. The model itself does not change.
- Good for facts that change: prices, policies, product docs.
- Answers can point back to the source they came from.
- Update the documents and the answers update with them.
Fine-tuning changes how the model behaves
Fine-tuning trains the model further on examples of the output you want. It is good at teaching a format, a tone or a narrow skill. It is a poor way to teach facts, because those facts get baked in and go stale.
- Good for consistent structure, house style or specialist classification.
- Needs a solid set of high-quality examples.
- Every change means another training run.
A simple rule of thumb
If the problem is "it does not know X", start with RAG. If the problem is "it knows enough but answers in the wrong way", look at fine-tuning. Plenty of production systems end up using both: RAG for knowledge, a light fine-tune for behaviour.
RAG is for knowledge. Fine-tuning is for behaviour.
In most projects we start with RAG and good prompts, measure where answers fall short, and only reach for fine-tuning when the gap is clearly about behaviour rather than knowledge.
Written by Agentive Team