What a context window actually is

“Context window” is one of those terms that gets used constantly by people who have never had it explained. It is worth ten minutes, because a surprising number of confusing behaviors become obvious once you understand it.

The simplest accurate description

A context window is a model’s token budget for a request, including the supplied context and room for the response. Depending on the model, reasoning tokens also count. Input and output can have separate limits, and multimodal inputs have their own accounting.

Tokens are pieces of encoded input or output, not a fixed number of words. English prose is sometimes estimated at about three-quarters of a word per token, but language, code and other media change the ratio; count with the model’s supported tools when a limit matters.

The application assembles or manages context for each request. Providers may reuse cached computation or stored conversation state, so this does not mean every visible word is freshly processed or charged at the same rate each time.

Why “it forgot” is the wrong mental model

People describe these systems as forgetting, and it leads to the wrong expectations in both directions.

A new conversation has no inherent access to an old one. The product can supply past information through memory, history retrieval, project files or connected tools, depending on its settings and permissions.

Earlier messages can influence a later response when they remain in context or are retrieved or summarized. Availability is not the same as attention or correct interpretation; an old instruction can be missed even when it is present.

A lost detail may result from truncation, summarization, retrieval failure or the model’s interpretation. Inspect what was available before assuming the problem was only the window limit.

What is in there that you did not put there

Most users assume the context window contains their conversation. In any real system it contains considerably more.

Depending on the application, context can include system instructions, tool definitions, project instructions, attachments, retrieved passages and prior messages or summaries. Their arrangement and visibility vary.

What you type is only part of the request. Tool definitions, system or developer instructions, retrieved material and prior turns may also take space; their share varies by application and task.

Attention is not evenly spread

Here is the part that is least known and most useful.

Research has found that some models perform worse when relevant evidence is in the middle of long inputs. That is a task- and model-dependent result, not a universal rule assigning more weight to the ends of every context window.

Put the task and important constraints where they are easy to identify, and clearly delimit source material. Test ordering on the model and task you use rather than assuming one prompt layout always wins.

If a constraint is ignored, check whether it was supplied, retained, contradicted or misunderstood. Moving it may help, but is not a complete diagnosis or a reliable control.

What happens when it fills up

Something has to give, and how the system handles it varies.

Dropping or summarizing history can lose details; rejecting an oversized request requires the caller to reduce it. Inspect the application’s documented behavior and any notices rather than assuming every strategy is silent.

Products differ in how they handle a full window: some reject the request, truncate, summarize or show a compaction notice. Check the behavior of the interface or API you are using.

A fresh conversation with a verified brief can help when a thread has accumulated irrelevant or conflicting material. Preserve the facts you still need, and compare the result instead of assuming a restart is always better.

More is not free

Three costs to spending context, all of which people underestimate.

Repeated history can consume input tokens, but prompt caching may change processing time and charges. Check usage records and the model’s current cache rules, including write and read charges where applicable.

More context can increase latency and cost, but caching, model behavior and output length also matter. Measure the whole request rather than predicting latency from visible length alone.

Supply the material needed to answer accurately. A short brief may outperform a pile of irrelevant documents, but shortening away necessary evidence can make the result worse.

The practical rules

Everything above reduces to a handful of habits.

Consider a fresh brief when a thread stalls. Carry over verified facts and constraints. For a task that depends on history, preserving the correct state may be better than restarting.

For a long document, try a clearly marked source followed by a precise question and requested evidence. Compare with other layouts on your task; position effects are not a fixed rule.

Restate what must not be forgotten. If a constraint genuinely matters and the conversation is long, say it again. It costs a sentence.

Do not confuse presence with correct use. Check that requirements are actually applied. Clear placement can help, but correctness still needs verification.

Memory features do not change the underlying rule

Several assistants now offer memory — things they recall about you across conversations. This does not contradict anything above; it is built on top of it.

Product memory is an application feature that selects prior information for a response. It may use saved facts, chat history, files or retrieval; it is not necessarily a single document inserted at the start.

Saved or inferred context can be wrong or stale. Review the controls and sources the product makes available; information may be selected automatically or supplied and corrected by you.

Review the memory and source controls your product exposes, and correct or remove stale information. User-edited and automatically inferred context may coexist, and the visible summary may not show every contributing source.

Why this is worth understanding

Context is one useful place to investigate a surprising answer, alongside model limitations, source quality and tool behavior. A correct diagnosis requires checking the actual request and evidence.

The practical habit is to manage relevant context deliberately and verify the resulting answer. Keep a concise, accurate brief when useful, but check the specific product’s persistence, limits and settings.

Technical review, October 10, 2026. References: OpenAI: conversation state; OpenAI: prompt caching; OpenAI: memory in ChatGPT.

Try this

Use a synthetic document with known facts in several positions. Ask the same specific questions across repeated runs, then compare accuracy while keeping other settings fixed. Record the model and whether the document was truncated or summarized; a single conversation cannot reveal a universal attention pattern.