What it is
Fine-tuning continues the training of a pre-trained model on a smaller set of examples, to specialise it. Alignment is the broader goal of making a model do what people actually want and avoid harm. For chat assistants this is often done by training on human-written answers and on human rankings of answers, as in InstructGPT. LoRA is a cheap way to fine-tune: it trains only a few small added matrices and leaves the original weights untouched.
What it is for
Making a model follow a format or tone consistently, teaching it a specialised task or vocabulary, shrinking the cost of a task by specialising a small model, and making assistants more helpful and safer. It is not the right tool for adding fresh facts: for that, retrieval is usually better.
How to do it
- First try prompting and retrieval of your documents; fine-tune only if they are not enough.
- Collect examples of the exact behaviour you want, as input and ideal-output pairs, and keep quality above quantity.
- Hold out some examples for testing, and write down how you will judge whether it got better.
- Choose the method: full fine-tuning when you have the resources, or LoRA when you want to train on modest hardware.
- Train, then compare against the original model on your held-out examples and on a few general questions to catch things it forgot or broke.
- Deploy the tuned model, keep the data and settings recorded, and plan to redo it when the base model changes.