inference
To begin with
When you ask an assistant a question, the model relearns nothing: it uses what it has already learned to compute an answer. That is inference.
The facts
During inference, the model computes an output from the inputs and its weights, without changing them. A language model generates its answer token by token and keeps a KV cache to avoid redoing some calculations. What you add to the conversation changes the context of the answer, not the weights.
Key takeaway
Inference uses the model; training changes it. A conversation does not retrain the model.
The stories about it
The notebook opens soon.The first stories are on their way.