The paper explains in-context learning—how an LLM can learn a task from examples in the prompt without permanently changing its trained parameters. Click here to read detailed review.
Its main argument is that adding each new prompt token is mathematically equivalent, within a contextual block, to making a tiny rank-1 update to the weight matrix of the following MLP. So although no optimizer or actual fine-tuning is run, the model behaves as though it is performing temporary, on-the-fly learning from the context.
Examples placed earlier in the prompt mainly teach the general task or mapping, while the final query specifies the particular input. The model has no special mechanism that distinguishes “examples” from “the question”; it simply processes one continuous sequence and follows the learned pattern created by prompt structure and position.