← The glossary

fine-tuning

To begin with

Instead of training a model from scratch, you start from a model that already knows a lot and adjust it for a task, a domain or a way of answering.

The facts

Fine-tuning is a new round of training that starts from already learned weights. It can change all the parameters, or only small added parts such as a LoRA adapter. It should not be confused with resuming an interrupted run, which also restores the optimizer state from a checkpoint. The result is measured on examples not used for training.

Key takeaway

Fine-tuning and resuming both start from existing weights: same mechanics, but a different goal and a different restored state.

The stories about it

The notebook opens soon.The first stories are on their way.