What Is Context Engineering?

If you've spent any time building with large language models, you've hit the wall.

Your prompt works perfectly in the playground. You ship it to production. And then it falls apart — not because the model is bad, but because the context is wrong.

The problem isn't the model. It's the information.

Every time you call an LLM, you're making a decision about what information to include. The system prompt. The user's message. Retrieved documents. Tool results. Conversation history. Few-shot examples.

Most teams treat this as an afterthought. They stuff everything into the prompt, hit the token limit, and start randomly truncating. The result? Inconsistent outputs, hallucinations, and agents that work 80% of the time — which, in production, means they don't work.

Context engineering is the discipline of being intentional about this. It's the practice of architecting what information AI systems see, when they see it, and how it's structured.

It's not prompt engineering

Prompt engineering is a subset of context engineering. Writing a good system prompt is important, but it's one piece of a much larger puzzle.

Context engineering encompasses:

• Information architecture — What data exists, how it's structured, and how it relates to the task at hand
• Retrieval design — How you find and select relevant information at runtime
• Token budgeting — How you allocate your finite context window across competing information needs
• Temporal management — How context evolves across multi-turn conversations and agent loops
• Evaluation — How you measure whether your context is actually helping or hurting

Why it matters now

Three trends are making context engineering increasingly critical:

1. Agents are multi-step. A simple chatbot has one context decision. An agent that runs for 20 turns has 20 context decisions, each building on the last. Small context errors compound.

2. Context windows are growing, but so is the data. GPT-4's 128K context window sounds enormous until you realize that a typical enterprise knowledge base has millions of tokens of potentially relevant information. Bigger windows don't solve the selection problem.

3. The gap between demo and production is a context gap. When your agent works in testing but fails in production, the usual culprit is that production data looks different from test data. The context assembly pipeline wasn't designed for the real world.

The mental model

Think of context engineering as having three layers:

Static context — Information that doesn't change per request: system prompts, few-shot examples, tool schemas, behavioral guidelines.

Dynamic context — Information assembled at runtime: retrieved documents, user history, current state, tool results.

Emergent context — Information generated during execution: chain-of-thought reasoning, intermediate tool outputs, agent scratch pads.

Good context engineering means designing systems that manage all three layers effectively, within the constraints of token budgets and latency requirements.

Getting started

If you're building AI systems and haven't thought explicitly about context engineering, start with an audit:

1. Map your context sources. What information goes into each LLM call? Where does it come from?
2. Measure your token budget. How much of your context window is static vs. dynamic? Are you leaving room for the model to think?
3. Test context ablation. Remove each context source one at a time. Does output quality change? You might be surprised by what's helping and what's hurting.

Context engineering is a young discipline, but it's quickly becoming the most important skill in production AI. The teams that master it will build AI systems that actually work — not just demos that impress.

---

This is the first in a series on context engineering. Follow along as we dive deeper into each layer — from retrieval design to token budgeting to evaluation frameworks.