Glossary
every word explained in the notebook
Every word in this list has its own short definition and leads to the stories that explain it. The glossary grows with the notebook.
-
architecture
A model’s computational design: its layers, connections, and operations. Two models can share this design while having different learned weights.
-
backpropagation
During training, the way back: the error is measured at the output and sent back layer by layer, to work out how to adjust…
-
calibration
In some quantization methods, a pass over representative data to choose how to reduce precision. This step alone is not fine-tuning for a new…
-
checkpoint
A saved snapshot of a model at a particular point. It may contain only the weights. Resuming the same training run also requires its…
-
computation graph
A description of computational operations and the data flowing between them. ONNX can store this graph so a compatible runtime can execute the model.
-
conversion
A transformation of a model’s representation for another tool or engine, such as an ONNX export. It does not teach the model a new…
-
expert
In a mixture-of-experts model, a small block specialised in one part of the work. It only works when the router picks it.
-
fine-tuning
Additional training that starts from an already trained model to adapt it. It may update many parameters or just a small subset, for example…
-
inference
The moment the model answers: it uses what it has learned, without changing anything inside itself.
-
KV cache
A working memory: while it writes, the model keeps calculations it has already done on the previous words (keys and values), so it doesn’t…
-
layer
A step of calculation. A model stacks dozens of them, and each token goes through them one by one, like the floors of a…
-
LoRA adapter
A small set of parameters added to adapt a model through training. The base weights usually stay frozen. An adapter distributed on its own…
-
model distribution
A set of files prepared to use a model in an environment: weights, configuration, tokenizer, and sometimes a computation graph. A compatible runtime is…
-
pretraining
An initial large training phase that builds a model’s basic capabilities. For a language model, it often uses large amounts of text to learn…
-
quantization
A way to represent some model values with lower precision, often to save memory. It makes the model smaller with a quality trade-off to…
-
router
In a MoE layer, the component that chooses which experts process each token. Its learned parameters are saved with the model weights; its choices…
-
token
A small piece of text: a word, part of a word or a punctuation mark. An AI doesn’t read whole sentences; it reads tokens,…
-
training
A process that adjusts a model’s parameters using examples and a measurable objective. These adjustments change how the model computes its answers.
-
weights
The numerical coefficients that control a model’s computations. Training adjusts some of them, and checkpoints preserve them. In a MoE model, they also include…