weights
To begin with
People sometimes imagine that a model contains texts it rereads. In fact, it mostly contains numbers: billions of settings, the weights.
The facts
Weights are the numerical coefficients that set the computations of each layer. Training adjusts them; a checkpoint saves them, often in Safetensors files. In a MoE model, they also include the learned coefficients of the router. What the model “knows” shows in its behaviour, not in a list of documents.
Key takeaway
Weights are learned numbers, not a library of texts. They do not change when the model answers.
The stories about it
The notebook opens soon.The first stories are on their way.