← The glossary

weights

To begin with

People sometimes imagine that a model contains texts it rereads. In fact, it mostly contains numbers: billions of settings, the weights.

The facts

Weights are the numerical coefficients that set the computations of each layer. Training adjusts them; a checkpoint saves them, often in Safetensors files. In a MoE model, they also include the learned coefficients of the router. What the model “knows” shows in its behaviour, not in a list of documents.

Key takeaway

Weights are learned numbers, not a library of texts. They do not change when the model answers.

The stories about it

The notebook opens soon.The first stories are on their way.