architecture
To begin with
Two models can share the same shape and still answer very differently. The shape is the architecture: the building’s plan, not what was put inside.
The facts
The architecture describes a model’s layers, their order, their connections and their operations: attention, computation blocks, normalisation. It also sets choices such as the number of layers or, in a MoE, the number of experts. It is often written in a configuration file. The learned values are in the weights.
Key takeaway
The architecture is the plan; the weights are what training has tuned. You need both to get a working model.
The stories about it
The notebook opens soon.The first stories are on their way.