checkpoint
To begin with
Training can last for weeks. So a snapshot of the model is taken regularly, like a video game save, to be able to resume or compare.
The facts
A checkpoint records a model’s state at a given moment. It may contain only the weights: that is what you download to use a model or start fine-tuning. A resume checkpoint also holds the optimizer state, the progress and the random states, to continue the same training run after an interruption.
Key takeaway
A checkpoint is a snapshot of the model’s state. It does not contain the training data.
The stories about it
The notebook opens soon.The first stories are on their way.