ai-agentscontext-engineeringmemory-managementarchitecturetutorial

Context Engineering for AI Agents: Architecture Guide

September 14, 2026

Context Engineering for AI Agents Architecture and Memory Management

The Context Problem: Why Multi-Turn Agent Loops Degrade and Fail

Autonomous AI agents—systems that plan, invoke tools, inspect observations, and self-correct across multi-turn loops—are transforming software automation. However, as agent workflows execute increasingly complex tasks (such as crawling dynamic web pages, reconciling multi-table databases, or auditing codebase repositories), developers encounter a critical bottleneck: context window degradation.

In a naive agent loop, every tool invocation, terminal dump, raw DOM response, and reasoning trace is appended monotonically to the conversation history. As execution progresses past 10, 20, or 50 turns, this unconstrained accumulation triggers severe operational failures:

Attention Dilution & “Lost in the Middle”: Foundation models do not allocate equal attention across hundred-thousand-token windows. When prompt length balloons, retrieval accuracy drops sharply for details located in the middle of the context, causing the agent to repeat failed tool calls or forget initial constraints.
Runaway Inference Latency & Cloud Costs: Time-to-First-Token (TTFT) scales quadratically or super-linearly with context length in standard attention mechanisms. Processing 150,000 tokens of accumulated web scrapes on every minor decision multiplies API costs and slows execution to a crawl.
Context Pollution & Poisoning: Unpruned error traces, stack dumps, and verbose HTML payloads distract the model from its primary goal. The model begins indexing its own previous missteps, falling into degenerative looping patterns.
Hard Context Window Overflow: Despite expanding context windows, long-running batch jobs and autonomous research tasks eventually breach maximum token thresholds, causing unhandled runtime exceptions.

Prompt engineering alone cannot solve this issue. Crafting the perfect system prompt does not protect an agent whose runtime memory is choked with 80,000 tokens of raw web markup. Solving this requires moving from static prompts to systematic Context Engineering.

The Solution: Architectural Principles of Context Engineering

Context Engineering is the discipline of actively curating, transforming, pruning, and compacting the information within an LLM working context window at every step of execution. Rather than treating the prompt as a static append-only transcript, context engineering treats the context window as high-speed, volatile RAM that must be actively managed by an operating system.

Context Engineering Architecture: Tiered Memory, Compaction, and Observation Masking

Prompt Engineering vs. Context Engineering

Dimension	Prompt Engineering	Context Engineering
Operational Phase	Design time (offline)	Runtime (dynamic per step)
Primary Focus	Phrasing, persona, and few-shot examples	Token budgeting, state compaction, and memory tiering
Context Mutation	Static instructions across turns	Active pruning, compression, and replacement
Observation Handling	Appends raw tool responses	Masks, filters, and summarizes outputs before ingestion
State Persistence	Monolithic message arrays	Multi-tiered memory (Working, Episodic, Semantic)
Target Failure Mode	Misaligned tone or vague answers	Attention dilution, hallucination loops, and token exhaustion

The Three Core Tiers of Agent Memory

A production-grade context engineering architecture partitions information across three distinct layers:

Working Memory (In-Context RAM): The immediate, strictly bounded token budget presented to the foundation model for the next reasoning step. This contains the system prompt, high-level task goal, active plan scratchpad, and only the most recent relevant observations.
Episodic Memory (Local Session Ledger): A structured historical log (typically stored in a local SQLite database) recording every raw tool call, complete API response, execution timestamp, and outcome. Working memory links to episodic records via unique IDs without keeping full payloads in context.
Semantic Memory (Long-Term Knowledge Store): Vector embeddings and indexed documentation retrieved just-in-time via similarity search, ensuring only query-relevant facts enter the working window.

Key Context Engineering Primitives

Observation Masking & Schema Projection: Tools should never dump raw HTTP responses or verbose HTML into the context. Structured extraction tools like AiFetchly Data Fetching prune extraneous DOM attributes, comments, and styles before returning clean JSON payloads.
Working Memory Compaction (Rolling Summarization): When working memory reaches a pre-set watermark (e.g., 70% of the allocated token budget), a background compaction pass synthesizes completed milestones into a concise state summary and evicts stale raw exchanges.
Scratchpad State Pruning: Instead of preserving every failed attempt, agents maintain an explicit plan or state object in context. Only current task status, verified findings, and pending steps remain active.

Implementation: Building a Resilient Context Compaction Pipeline

Let us implement a production-ready context engineering manager in TypeScript. This implementation demonstrates token budget monitoring, observation pruning, and automatic context compaction for multi-turn agent loops.

Step 1: Define Memory Interfaces and Token Budgets

Establish explicit token limits and data structures for working memory and execution turns:

export interface AgentTurn {
  id: string;
  role: 'user' | 'assistant' | 'tool';
  content: string;
  tokenCount: number;
  timestamp: number;
  isPinned?: boolean;
}

export interface ContextConfig {
  maxContextTokens: number;
  compactionThresholdTokens: number;
  reservedOutputTokens: number;
}

export const DEFAULT_CONFIG: ContextConfig = {
  maxContextTokens: 32000,
  compactionThresholdTokens: 24000, // Trigger compaction at 75% capacity
  reservedOutputTokens: 4000,
};

Step 2: Implement Observation Pruning & Schema Filtering

Before passing external data to the agent, sanitize and filter large tool outputs:

export function pruneToolObservation(
  rawPayload: Record<string, unknown>,
  allowedKeys: string[],
): string {
  const sanitized: Record<string, unknown> = {};

  for (const key of allowedKeys) {
    if (key in rawPayload) {
      sanitized[key] = rawPayload[key];
    }
  }

  const jsonString = JSON.stringify(sanitized);

  // Truncate payloads exceeding safe single-turn limits
  const MAX_TOOL_CHARS = 4000;
  if (jsonString.length > MAX_TOOL_CHARS) {
    return jsonString.slice(0, MAX_TOOL_CHARS) + '... [Truncated for Context Optimization]';
  }

  return jsonString;
}

Step 3: Implement the Context Engineering Manager

Build the stateful controller that evaluates token consumption and performs intelligent rolling compaction:

export class ContextManager {
  private systemPrompt: AgentTurn;
  private currentGoal: AgentTurn;
  private turns: AgentTurn[] = [];
  private config: ContextConfig;

  constructor(systemPromptText: string, goalText: string, config = DEFAULT_CONFIG) {
    this.config = config;
    this.systemPrompt = {
      id: 'system',
      role: 'user',
      content: systemPromptText,
      tokenCount: this.estimateTokens(systemPromptText),
      isPinned: true,
    };
    this.currentGoal = {
      id: 'goal',
      role: 'user',
      content: `Current Objective: ${goalText}`,
      tokenCount: this.estimateTokens(goalText),
      isPinned: true,
    };
  }

  public estimateTokens(text: string): number {
    // Standard rule of thumb: ~4 characters per token
    return Math.ceil(text.length / 4);
  }

  public getTotalTokens(): number {
    const turnsTokens = this.turns.reduce((acc, turn) => acc + turn.tokenCount, 0);
    return this.systemPrompt.tokenCount + this.currentGoal.tokenCount + turnsTokens;
  }

  public async addTurn(role: 'assistant' | 'tool', content: string): Promise<void> {
    const turn: AgentTurn = {
      id: `turn-${Date.now()}-${Math.random().toString(36).slice(2, 7)}`,
      role,
      content,
      tokenCount: this.estimateTokens(content),
    };

    this.turns.push(turn);

    if (this.getTotalTokens() > this.config.compactionThresholdTokens) {
      await this.compactContext();
    }
  }

  private async compactContext(): Promise<void> {
    // Preserve the 4 most recent turns for direct conversational continuity
    const turnsToKeep = 4;
    if (this.turns.length <= turnsToKeep) return;

    const olderTurns = this.turns.slice(0, this.turns.length - turnsToKeep);
    const recentTurns = this.turns.slice(this.turns.length - turnsToKeep);

    // Synthesize older history into an active scratchpad summary
    const summaryText = olderTurns
      .map((t) => `[${t.role.toUpperCase()}]: ${t.content.slice(0, 120)}...`)
      .join('\n');

    const compactedTurn: AgentTurn = {
      id: `summary-${Date.now()}`,
      role: 'user',
      content: `[Previous Turns Summary & State Checkpoint]:\n${summaryText}`,
      tokenCount: this.estimateTokens(summaryText),
      isPinned: false,
    };

    this.turns = [compactedTurn, ...recentTurns];
  }

  public getWorkingContext(): { role: string; content: string }[] {
    return [
      { role: this.systemPrompt.role, content: this.systemPrompt.content },
      { role: this.currentGoal.role, content: this.currentGoal.content },
      ...this.turns.map((t) => ({ role: t.role, content: t.content })),
    ];
  }
}

Step 4: Execute Agentic Workflows with Bounded Memory

Integrate the ContextManager directly into an autonomous execution loop:

async function runAutonomousTask(objective: string): Promise<void> {
  const contextManager = new ContextManager(
    'You are an autonomous research agent. Maintain an accurate state scratchpad.',
    objective,
  );

  let taskCompleted = false;
  let iterations = 0;
  const MAX_ITERATIONS = 20;

  while (!taskCompleted && iterations < MAX_ITERATIONS) {
    iterations++;
    const promptPayload = contextManager.getWorkingContext();

    // Call your LLM inference runtime with strict token budget
    console.log(`Iteration ${iterations} | Active Tokens: ${contextManager.getTotalTokens()}`);

    // Mock agent tool call & sanitized response
    const mockToolOutput = pruneToolObservation(
      { status: 'ok', recordsFound: 142, debugData: 'trace_stack_bytes' },
      ['status', 'recordsFound'],
    );

    await contextManager.addTurn('assistant', 'Extracting targeted metrics from source.');
    await contextManager.addTurn('tool', mockToolOutput);

    if (iterations >= 6) {
      taskCompleted = true;
    }
  }

  console.log('Task completed within token budget.');
}

Production Best Practices for Context Engineering

When deploying autonomous agents in production environments:

Enforce Hard Token Caps per Tool: Never allow any single tool execution to return more than 10% of your maximum context budget. Require large queries to paginate or stream to disk.
Decouple Data Storage from Context: Store full web pages and raw JSON documents in local storage (such as SQLite or flat files). Inject only targeted extracts or high-confidence summaries into the working prompt.
Maintain Deterministic Scratchpads: Provide a dedicated system structure (e.g., Goal, Completed Steps, Next Action, Blockers) so the agent continuously overwrites its state rather than appending historical chatter.
Log Full Transcripts Out-of-Band: Keep comprehensive telemetry in persistent logs for auditing and debugging, while keeping runtime agent memory lean and focused.

Conclusion

Building reliable AI agents requires moving beyond prompt tuning to deliberate context engineering. By applying observation masking, rolling working memory compaction, and clear boundaries between volatile working context and persistent episodic storage, software engineers can prevent attention dilution, curb runaway token costs, and build agents capable of executing hundreds of autonomous reasoning steps without degradation.