LoRA adapter
To begin with
Adapting a large model is expensive if you change all of its weights. LoRA leaves the model as it is and adds a small custom part, like a lens clipped onto a camera.
The facts
During fine-tuning, LoRA freezes the base weights and trains, in some layers, two small matrices whose product forms a correction. They often amount to a small fraction of the parameters. The resulting file is light, but it only works with the base model it was trained for. It can also be merged into that model.
Key takeaway
A LoRA adapter is a learned correction, not a complete model. Without its base, it is useless.
The stories about it
The notebook opens soon.The first stories are on their way.