← The glossary

architecture

To begin with

Two models can share the same shape and still answer very differently. The shape is the architecture: the building’s plan, not what was put inside.

The facts

The architecture describes a model’s layers, their order, their connections and their operations: attention, computation blocks, normalisation. It also sets choices such as the number of layers or, in a MoE, the number of experts. It is often written in a configuration file. The learned values are in the weights.

Key takeaway

The architecture is the plan; the weights are what training has tuned. You need both to get a working model.

The stories about it

The notebook opens soon.The first stories are on their way.