RAG vs fine-tuning for business knowledge

Retrieval-augmented generation, or RAG, supplies relevant source material when a model answers. Fine-tuning changes a model through additional training examples. For frequently changing business facts, retrieval is often the more direct place to start. Fine-tuning can address repeatable behavior or format needs, but it is not a substitute for current records and access control.

Separate facts from behavior

Imagine an assistant answering questions about a service catalog. Prices and availability change. Those facts should come from the current catalog. The desired answer structure may remain stable: explain the option, list exclusions and identify the next step. Instructions, examples or training can influence that behavior.

Confusing the two leads teams to train on documents when they actually need a reliable lookup, or to add more documents when the response pattern is the problem.

Inspect a failed answer

Ask whether the correct source was available, whether retrieval selected it and whether the model used it faithfully. Only after those checks should you conclude that the model itself needs adaptation. A missing access filter or stale index will not be repaired by more training examples.

For a retrieval test, include a document the requester cannot access. The system must respect that boundary before providing text to the model, not merely promise to avoid mentioning it afterward.

Compare effort with a small experiment

• Define a narrow set of questions and the expected evidence for each.
• Try instructions plus retrieval using authoritative documents.
• Record failures as missing knowledge, retrieval errors or behavior errors.
• Improve the relevant layer and rerun the same cases.
• Consider fine-tuning only for a remaining, measurable need.

Keep a held-out test set so improvements are not judged only on the examples used during development.

Avoid the perfect-memory promise

Neither approach makes an agent know every company fact. Both need maintenance, evaluation and a path for uncertainty. Ask the system to show the source behind consequential answers and to say when the available evidence does not settle the question.

Be Human editorial guidance. Examples are illustrative unless explicitly attributed. Sources provide background; implementation choices depend on your workflow.