The Humanevi newsletter
Weekly
Once a week: 5 things worth your time.
1 story, 1 idea, 1 discovery, 1 tool, 1 book.
The Humanevi newsletter
Once a week: 5 things worth your time.
1 story, 1 idea, 1 discovery, 1 tool, 1 book.
When you talk to a chatbot, it seems to remember everything said. In practice, what it considers with each answer is limited and depends on what is in the context. Understanding this helps explain why, sometimes, the assistant "forgets" something, contradicts itself or answers unexpectedly.
Context is the set of text the model receives and considers when generating an answer. It usually includes the initial instructions, your messages, previous answers and, when present, documents or tool results you attached. The model does not "read your mind" or keep, on its own, what you said in separate conversations, unless the service offers a memory feature.
Each model has a maximum context size, measured in units called tokens, which are pieces of text. When the conversation gets too long, older parts may fall out of the model's reach or be summarized by the system. That is why, in very long conversations, the assistant may lose details from the beginning. Even within the limit, important information in the middle of a very long text may receive less attention.
Camila asked a chatbot to write a formal email, with a cordial tone and at most five lines. After many exchanges, the assistant started replying with long, informal texts. When she repeated the instructions at the start of a new message, it went back to the requested format. This is an illustrative example.
Context is also a matter of trust: what you include in it may be stored or processed by the service. Before pasting personal or professional information, check the service's policy.
