fine-tuning
To begin with
Instead of training a model from scratch, you start from a model that already knows a lot and adjust it for a task, a domain or a way of answering.
The facts
Fine-tuning is a new round of training that starts from already learned weights. It can change all the parameters, or only small added parts such as a LoRA adapter. It should not be confused with resuming an interrupted run, which also restores the optimizer state from a checkpoint. The result is measured on examples not used for training.
Key takeaway
Fine-tuning and resuming both start from existing weights: same mechanics, but a different goal and a different restored state.
The stories about it
The notebook opens soon.The first stories are on their way.