Operations
Fine-tuning
Definition
Fine-tuning adapts a base model to your style, format, or task using your own examples — powerful for consistency, largely useless for supplying facts.
Teams reach for fine-tuning to make a model 'know our business' and are disappointed, because knowledge belongs in retrieval and behaviour belongs in fine-tuning, and conflating the two produces a model that is confidently wrong about facts it was never a good tool for storing. Where it genuinely earns its cost is enforcing a house format, a classification taxonomy, or a tone across very high volume — and in letting a smaller, cheaper model match a larger one on a narrow task, which is a real and durable economic win when it applies.
Behaviour, not knowledge
Use retrieval for what is true and fine-tuning for how the answer should look, because these are genuinely different problems that fine-tuning alone cannot solve for the knowledge side. Facts baked into weights go stale and cannot be cited, which is a serious liability in any regulated workflow that needs to show its work, whereas a well-designed retrieval layer can be updated the moment a source document changes, with the update reflected in the very next call the system makes.
It creates a maintenance obligation
A fine-tuned model is an asset you now own: re-training when the base changes, re-validation against the golden dataset, and version control across every iteration you ship. Budget for the lifecycle, not the training run, because the training run itself is often the cheapest part of the total cost, and teams that budget only for it are routinely surprised by the ongoing cost of keeping a fine-tuned model current as the underlying base model and the business both evolve.
Know when not to fine-tune at all
A large share of use cases that teams initially propose for fine-tuning can be solved more cheaply and more maintainably with better prompting or better retrieval alone, and skipping straight to fine-tuning before exhausting those options is a common and expensive mistake. Fine-tuning earns its cost specifically at high volume, on a narrow and stable task, where the ongoing maintenance burden is justified by savings large enough to clearly outweigh it — and it should be evaluated against that specific bar before committing.
In practice
- Exhaust prompting and retrieval before spending on fine-tuning
- Use it to shrink model size at constant quality, and measure the saving
- Keep evaluation data strictly separate from training data
- Budget the ongoing re-training and re-validation lifecycle, not just the initial run
