Context Engineering
The discipline of curating exactly what goes into an LLM's limited context window — the successor to prompt engineering.
“Prompt engineering” was about wording a single request well. But an agent’s performance is governed by something bigger: everything in its context window at the moment it acts — the system prompt, tool definitions, retrieved data, conversation history, and memory. Context engineering is the discipline of curating that whole set so the model has exactly what it needs and nothing that distracts it. As agents run longer and call more tools, it has quietly become the skill that separates a flaky agent from a reliable one.
Why it matters: context is a finite, degrading budget
A context window is large but not free, and not uniformly effective:
- It’s finite — every token of tools, history, and data competes for the same space.
- It degrades — models attend less reliably to information buried in the middle of a very long context (the “lost in the middle” effect), so more context can mean worse answers.
- It costs — tokens are money and latency; a bloated context is slow and expensive on every single turn.
The goal isn’t to fill the window — it’s to find the smallest set of high-signal tokens that maximizes the odds of a good next step.
The context budget
Think of the window as a budget you allocate:
The core techniques
1. Curate what goes in
- Retrieve, don’t dump. Pull in only the relevant passages via RAG instead of pasting whole documents.
- Prune tools. Ten sharp tools beat fifty vague ones — every tool definition costs tokens and adds a chance for the model to pick wrong.
- Right-size the system prompt. Specific and minimal. Long lists of rules dilute the important ones.
2. Compact as you go
Long agent loops accumulate history until they overflow. Manage it:
- Summarize / compact. Periodically replace old turns with a concise summary of what happened and what matters going forward.
- Structured note-taking. Have the agent write key facts to an external scratchpad (a file, a memory store) and reload only what’s needed — durable memory that lives outside the window.
- Prune tool results. A tool that returned 10,000 rows doesn’t need all of them in context for the next step; keep the summary, drop the raw dump.
3. Isolate with sub-agents
Hand a self-contained sub-task to a subagent with its own fresh window. It does the deep work and returns just the result — the messy intermediate context never pollutes the main agent’s window. Context isolation is one of the strongest reasons to go multi-agent.
Memory: context that outlives the window
Because the window resets, lasting knowledge has to live outside it and be retrieved back in when relevant:
| Memory type | Lives in | Example |
|---|---|---|
| Working (short-term) | The window itself | The current conversation |
| Long-term | External store (vector DB, files) | User preferences, past decisions |
Good agents write to long-term memory deliberately and pull only the relevant pieces back — the same retrieve-only-what-you-need principle, applied across time.
The mindset shift
Prompt engineering asks “what do I say?” Context engineering asks “what should be in the model’s head right now — and what should not?”
For anything beyond a single-shot prompt — agents, long conversations, tool-heavy workflows — this framing is the one that scales. Treat the context window as the scarce resource it is, and spend it deliberately.