Language models are frequently described as remembering a conversation. What they actually do is reread it, and the distinction accounts for most of the surprising behaviour users encounter.
The window is a reading limit
A model processes a block of text and produces a continuation. Everything it can consider must be inside that block, including the instructions, the conversation so far and any documents supplied.
Each new message causes the whole accumulated block to be processed again from the beginning. Nothing carries over from the previous turn except the text itself.
When the block exceeds the limit, the oldest material is dropped or condensed. From the user's side this appears as the model forgetting something it clearly knew earlier.
Persistence is a separate system
Products that recall details between sessions do so by storing those details externally and inserting them into the block at the start of each conversation.
The model has no access to that store and no awareness of it. It sees whatever was retrieved and placed in front of it, exactly as if the user had typed it.
This is why such recall can be inconsistent. The retrieval step decides what to include, and a detail that was not selected is simply absent rather than forgotten.
Attention is not uniform across the window
A model can technically consider everything in a long block, but material in the middle of a very long input tends to influence the output less than material near either end.
Instructions given early in a long conversation therefore lose force as the conversation grows, without any explicit point at which they were dropped.
Restating important constraints periodically works for this reason, and it is more reliable than assuming an instruction from many turns ago still carries full weight.
Long contexts are expensive
Processing cost grows with the amount of text considered, and it grows faster than linearly for the attention component of the computation.
A long conversation therefore costs more per message than a short one, even when the new message is a single line.
This is why products summarise older turns rather than carrying the full history, trading fidelity for cost and speed in a way the user does not see.
Larger windows do not remove the design problem
Expanding the window makes more material available but does not make the model better at selecting which parts matter.
Filling a large window with everything that might be relevant often produces worse results than supplying a smaller, carefully chosen set of material.
The work of deciding what the model should see remains the determining factor, and it does not become less important as the limits rise.