From Prompt Engineering to Context Engineering: Why Context Is the New Lever for Enterprise AI

Most failed AI projects don’t fail because of the model. They fail because the model sees the wrong information at the wrong time — or never receives the right information at all. Anyone who was still polishing the perfect prompt in 2024 is, today, optimizing something larger: the entire information environment in which an AI model operates. Context engineering is the name of this shift — and for businesses it is far more than a buzzword. Because the moment you deliberately shape the context, internal documents, customer data, and tool outputs all flow into the model. That inevitably turns a technical question into a data protection question. This article shows both sides: the architecture that works — and the legal obligations that ride along with it.

What is context engineering? (Definition)

Context engineering is the discipline of deliberately filling an AI model’s context window with exactly the right information for the next step. It covers not just the single input (the prompt), but everything else that ends up in the context: system instructions, retrieved documents, tool outputs, conversation history, and stored knowledge. The term was coined in 2025 by, among others, AI researcher Andrej Karpathy and Shopify CEO Tobi Lütke.

Karpathy described it as “the delicate art and science of filling the context window with just the right information for the next step.” Lütke framed it as “the art of providing all the context for the task to be plausibly solvable by the model.” Anthropic formalized the term in a widely cited engineering post on September 29, 2025, as “the strategies for curating and maintaining the optimal set of tokens during inference.”

What all belongs to an LLM’s context?

A language model’s “context” is not a single piece of text but a dynamically assembled bundle. It typically includes:

• System prompt / instructions — role, rules, tone.
• RAG / retrieval — relevant documents injected at runtime.
• Tools — defined functions the model is allowed to call (such as a database query).
• Memory — short-term conversation history and long-term, persistent knowledge.
• Examples — few-shot templates that guide the model.

Prompt engineering deals only with the first item. Context engineering orchestrates all of them.

Prompt engineering vs. context engineering — the difference

The shortest way to distinguish them: prompt engineering asks “How do I phrase the input?”, context engineering asks “What information does the model need to be able to access right now?” Prompt engineering is a one-off wording task; context engineering is an ongoing process that decides anew at every step what goes into the model.

Dimension	Prompt Engineering	Context Engineering
Subject	Single input (wording)	Entire information environment
Time horizon	Point-in-time, per request	Ongoing, across many steps
Persistence	Ephemeral	Persistent (memory, knowledge base)
Complexity	Low	System level (architecture)
Analogy	Writing down a good recipe	Setting up a fully equipped kitchen

An important point for context: prompt engineering is now a subset of context engineering, not its opposite. A precise prompt remains useful — it has simply become one lever among several.

Is prompt engineering dead?

No. The ability to instruct clearly remains relevant; it is just no longer sufficient. Industry surveys from 2026 make the shift clear: a majority of IT and data leaders consider prompt engineering alone inadequate for running AI “at scale,” and a large share of data teams plan to invest in context-engineering capability in 2026 (DataHub, 2026). Calling it “dead” is clickbait — “promoted to a sub-discipline” is closer to the truth.

Why the shift? The 2026 paradigm change

The bottleneck in modern enterprise AI is rarely the model itself. The top models are capable — they simply know nothing about your company. The real lever lies in the proprietary data around them: contracts, knowledge bases, tickets, internal policies. Anyone moving AI from prototype to production discovers that most failures in agentic systems are no longer model errors but context errors — the model received the wrong, outdated, or incomplete information.

That is precisely what marks the paradigm shift: away from “ask better,” toward “supply better.” RAG, tools, and memory are the mechanics of that supply.

The bottleneck moves: it is not model capability that decides whether an AI project succeeds, but the quality of the context you provide to the model — data, tools, and memory.

The new lever in detail — RAG, tools, memory, context window

RAG / retrieval — injecting knowledge at runtime

Retrieval-augmented generation (RAG) fetches matching documents from a knowledge base at query time and presents them to the model as context. This lets the AI answer based on current, company-owned content rather than from its training memory. RAG is powerful — but it is only one component of context engineering, not the whole. When to choose RAG and when fine-tuning is the right tool depends on the use case; for changing, citation-bound content RAG is almost always the more robust choice.

Tools — what the model is allowed to do

Tools are defined functions the model can call: a search, a calculation, a database access. Design is what counts here: according to Anthropic, an overloaded, ambiguous tool set leads to worse decisions than a few clearly delineated tools.

Memory — short-term and long-term

Memory distinguishes the conversation history (short-term) from the persistent knowledge (long-term) that lives, for example, in a vector database. One proven technique is structured note-taking: the agent records interim results outside the context window and retrieves them later as needed.

Managing the context window

The context window is the limited token capacity a model “keeps in view” per request. The windows have grown sharply of late — with Gemini 2.5 Pro, Google announced a one-million-token window on March 25, 2025. Yet it remains finite, and more content is not automatically better; why it remains a scarce resource despite growing window sizes is something I explore in the article on context windows and token limits. Common techniques for keeping it clean:

• Compaction / summarization — condense the history as it approaches the limit.
• Offloading — move knowledge out of context and pull it back only when needed.
• Isolation via sub-agents — specialized sub-agents work with a clean context and return only a distilled summary (often 1,000–2,000 tokens).

Why more context is not automatically better

A central piece of practical know-how: quality drops when the window overflows. Three failure modes businesses should know:

• Context rot — the accuracy of information retrieval declines as the window fills up (Anthropic, 2025).
• Context poisoning — a hallucination or piece of misinformation, once introduced, reproduces itself across subsequent steps.
• Context distraction / confusion — too much, partly contradictory content pulls the model away from the goal.

The consequence: context engineering is about curating, not accumulating.

What does this mean for businesses?

From one-off prompts to maintained context systems

A good prompt was a single artifact. A context system is infrastructure: it requires maintenance, approval processes, budget control for token costs, and auditable logs of which information the AI “saw” when it made a decision. Adopting RAG, memory, and tools also means taking on ongoing operational and governance effort — that is a deliberate investment decision, not a side effect.

Context is (often) personal data — the data protection dimension

This is the blind spot of most context-engineering articles. The moment RAG documents, memory contents, and tool outputs flow into the context window, personal or sensitive company data is, as a rule, being processed — and that is a processing operation within the meaning of the GDPR.

On October 17, 2025, the Datenschutzkonferenz (DSK, the conference of German data protection authorities) published a guidance note on the data protection particularities of generative AI systems using the RAG method. Key takeaways for practice:

• A legal basis under Art. 6 GDPR is required — a blanket appeal to “legitimate interest” does not automatically hold.
• Processing occurs in both the retriever and the LLM component; both must be considered.
• RAG does not cure the data protection problems of an unlawfully trained base model.
• Required technical and organizational measures include tenant separation, role-based access rights to the vector database, a data processing agreement (Auftragsverarbeitungsvertrag, Art. 28 GDPR), and, where applicable, a data protection impact assessment (Art. 35 GDPR).

In other words: whoever designs the context designs a data processing operation. The question “What may the AI see?” is always also a legal question. Whether a data protection impact assessment is mandatory in a given case depends on the scope and sensitivity of the data — for extensive or particularly sensitive holdings it is regularly indicated.

Governance and liability of the context

Who is accountable for what the AI “knows”? That question belongs in an AI usage policy. On top of this comes the AI literacy obligation: Art. 4 of the EU AI Act has been in force since February 2, 2025 and requires companies to ensure an adequate level of AI understanding among their staff (implementing Art. 4 AI Act in practice). National enforcement is expected to take effect from August 2, 2026.

To place the overall framework: possible shifts to individual deadlines for high-risk AI (Annexes I and III of the AI Act) are still being negotiated at EU level — the European Commission tabled a proposal to this end (the “Digital Omnibus on AI”) on November 19, 2025. What matters in practice: a postponed deadline changes nothing about the substantive obligations — postponed does not mean abolished. Since the timetable here is in motion (as of January 2026), you should check the current state against the official sources (EUR-Lex, DSK) before making concrete project decisions.

This article is general information, not legal advice for an individual case.

How does my company get started?

Pragmatically and with data protection in mind, in small steps:

1. Narrow the use case tightly — a clearly defined problem rather than “AI for everything.”
2. Inventory the data sources — what should the AI see, and does it contain personal data?
3. Clarify the legal basis before the technology is in place — the sequence determines audit-readiness.
4. Pilot with EU / on-prem data residency — keep the risk small, then scale.

This path — building RAG systems and AI agents so that they work technically and stay clean on data protection where the context is concerned — is one we walk at MusketierSoftware from a single source: a business lawyer and developer in one person.

FAQ

What is context engineering, simply explained?

Context engineering is the discipline of deliberately filling an AI model’s context window with the right information — from system instructions, retrieved documents (RAG), tools, memory, and history. The goal is to enable the model to solve a task reliably.

What is the difference between prompt engineering and context engineering?

Prompt engineering optimizes the single input (“How do I phrase it?”). Context engineering shapes the entire information environment (“What does the model need to be able to access?”). Prompt engineering is now a subset of context engineering.

Is context engineering the same as RAG?

No. RAG (retrieval-augmented generation) is an important component — injecting relevant documents at runtime. Context engineering additionally encompasses tools, memory, managing the context window, and avoiding failure modes such as context rot.

Why is more context not automatically better?

Because model quality drops when the context window is overstuffed. Phenomena such as context rot (declining retrieval accuracy), context poisoning (self-reproducing errors), and context distraction make curating more important than accumulating.

Is context engineering relevant under the GDPR?

Yes. The moment RAG documents, memory, or tool outputs contain personal data, processing under the GDPR is taking place. The DSK guidance note (October 2025) requires, among other things, a legal basis, a data processing agreement, technical measures such as tenant separation, and, where applicable, a data protection impact assessment. This is general information, not legal advice for an individual case.

Conclusion

The shift from prompt to context is not a hype term but a maturation: enterprise AI is won not through cleverer sentences but through cleanly designed, maintained, and legally sound information environments. Anyone who takes context engineering seriously treats it as what it is — architecture, operations, and data protection rolled into one.

Want to deploy AI in your business without running aground on data protection when it comes to context? Let’s talk about your specific use case in an initial consultation.

— Leon Lotz, business lawyer (MusketierSoftware)

---

Sources — as of 04.01.2026

• Anthropic — “Effective context engineering for AI agents” (09/29/2025): https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
• Andrej Karpathy, definition “delicate art and science of filling the context window” (X, 2025): https://x.com/karpathy/status/1937902205765607626
• Google — “Gemini 2.5: Our newest Gemini model with thinking” (1M-token context window, 03/25/2025): https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/
• DataHub — “Context Engineering vs Prompt Engineering” (industry survey 2026): https://datahub.com/blog/context-engineering-vs-prompt-engineering/
• Elastic — “Context engineering vs. prompt engineering”: https://www.elastic.co/search-labs/blog/context-engineering-vs-prompt-engineering
• DSK — “Orientierungshilfe: Datenschutzrechtliche Besonderheiten generativer KI-Systeme mit RAG-Methode” (10/17/2025): https://www.datenschutzkonferenz-online.de/orientierungshilfen.html
• SKW Schwarz — analysis of the DSK guidance note on RAG: https://www.skwschwarz.de/news/KI-Flash-Neue-Orientierungshilfe-der-DSK-zu-RAG-basierten-kI-Systemen
• Fraunhofer Academy — “AI Literacy: EU AI Act Art. 4 on AI competence” (in force since 02/02/2025): https://blog.academy.fraunhofer.de/blogbeitraege/ai-literacy/
• TÜV Rheinland Consulting — “Digital Omnibus on AI: AI Act deadlines” (on the Commission proposal of 11/19/2025): https://consulting.tuv.com/aktuelles/ki-im-fokus/digital-omnibus-ki-verordnung-fristen