← The glossary

LoRA adapter

To begin with

Adapting a large model is expensive if you change all of its weights. LoRA leaves the model as it is and adds a small custom part, like a lens clipped onto a camera.

The facts

During fine-tuning, LoRA freezes the base weights and trains, in some layers, two small matrices whose product forms a correction. They often amount to a small fraction of the parameters. The resulting file is light, but it only works with the base model it was trained for. It can also be merged into that model.

Key takeaway

A LoRA adapter is a learned correction, not a complete model. Without its base, it is useless.

The stories about it

The notebook opens soon.The first stories are on their way.