200K Context Window——AI's "Memory Palace" Management

Have you ever wondered: when you’ve been chatting with Claude Code for a long time, how does it remember what you said earlier?

Claude Code supports a 200K context window—that sounds huge—can fit 150,000 characters of text. But compared to large codebases, it’s nothing. A medium-sized project easily exceeds this number.

It’s like giving you a 200-square-meter study to fit books from an entire library. What do you do? The answer is “memory palace” management—what to discard, what to keep, what to box up.

The diagram: 200K context is like a memory palace, requiring careful organization and tradeoffs

200K Isn’t “Unlimited Memory”

First, clarify: 200K token is an upper limit, not “unlimited memory.”

What is a token? Simply put, it’s the basic unit AI processes text in. English words take roughly 1-2 tokens, Chinese characters roughly 1-3 tokens. 200K tokens can roughly fit:

• 150,000 English words
• 300,000 Chinese characters
• Or several hundred medium-length code files

In actual development:

• A React project might have thousands of files
• A large open source project might have hundreds of thousands of lines of code
• 200K can’t fit it all

So Claude Code can’t “remember” an entire codebase—it can only “remember” what’s processed in the current conversation. This brings us to the core question of context management: within a limited window, what to put in, what to discard, how to organize.

Auto Compaction: When Memory Isn’t Enough

When context approaches its limit, Claude Code automatically initiates compaction. This process is like packing up when moving: box up things you don’t use often, leaving only labels on the outside.

Compaction flow:

Detect Context Size → Exceeds Threshold → Identify Compressible Content → Execute Compaction → Update Context

What gets compacted?

Old Tool Results: Tool results from early conversation turns, if not referenced later, get compacted into summaries.

Code Blocks in History: Processed code, keeping only key parts.

Redundant Information: Repeated content, outdated state.

After compaction, the model sees not complete content, but “summary + references.” If detailed content is needed, it can be re-read (via references).

The diagram: Auto compaction flow, boxing old content and keeping labels

Microcompact: More Precise Pruning

Beyond large-block auto compaction, Claude Code also supports microcompact (Microcompact)—finer-grained context pruning.

Think of it this way:

• Auto compaction is like boxing up an entire book
• Microcompact is like pulling out certain chapters from a book

Microcompact application scenarios:

Individual File Content Too Large: A log file with tens of thousands of lines, but the model only needs a few relevant lines.

Tool Results Redundant: Grep returned 1000 matches, but the model only cares about the first 50.

Long Code Blocks in History: Preserve function signatures, compact implementation details.

Microcompact decisions have the model itself participating—it decides “what’s important.” This aligns with the “on distribution” philosophy: the model isn’t just passively receiving information, but actively managing its own context.

Token Budget Strategy: How to Spend Money Wisely

200K tokens is like a budget—spend it wisely.

Claude Code’s budget allocation strategy:

System Prompts: About 10-20% of total tokens. This is “fixed overhead,” includes tool definitions, behavioral norms, etc.

Current Conversation: About 30-40%. Recent turns are kept complete.

Historical Context: About 20-30%. Earlier conversation might be compacted.

Reserved Buffer: About 10-20%. Leave space for new messages, tool results.

The diagram: Token budget allocation strategy

This allocation isn’t fixed but dynamically adjusts:

• If tool results are large, compact historical context
• If conversation is short, keep more history
• If approaching limit, compact aggressively

File State Preservation: What Remembers After Compaction

A key question: after content is compacted, what does the model still “remember”?

Claude Code’s approach is “lossy compression”—preserve key information, discard details.

Specifically, compacted content retains:

Existence: “I read this file”

Key Metadata: File path, general content type, processing time

Relevance Markers: Which content relates to current task

Reference Links: Where to recover detailed content if needed

What’s compacted away:

Complete Content: Specific file content, complete tool results

Outdated Information: Content already overwritten by subsequent operations

Redundant Data: Repeated or irrelevant content

This is like your memory: you might remember “yesterday I read an article about Rust,” but not necessarily every sentence—if needed, you can go back and read.

Context Collapse: The Last Defense

When even after compaction the context still doesn’t fit, Claude Code deploys its final move: Context Collapse.

It’s like when moving and you really can’t fit everything—you have to throw away some boxes, keeping only the most important.

Collapse strategy:

Preserve Recent Conversation: Recent N turns remain complete.

Preserve System Prompts: This is AI’s “identity,” can’t be lost.

Compact or Discard Early Content: Earliest turns might be completely removed.

Preserve Key Decision Points: Important branch points, explicitly stated user requirements retain summaries.

After collapse, the model may “forget” early details. This is why in long conversations, AI sometimes asks “what was that you mentioned earlier”—it might have been collapsed away.

Practical: How to Manage Context

What does understanding context management help you with when using Claude Code?

Understand “forgetting” is normal. If a conversation is long, AI “forgetting” early content isn’t a bug—it’s the mechanism.

Proactively provide context. If AI seems to have forgotten earlier content, proactively remind it: “We talked about using React earlier, remember?”

Use CLAUDE.md to store key information. Project-level key information goes in CLAUDE.md—it gets priority preservation.

Split long tasks. If a task is very complex, divide it into multiple subtasks to avoid context explosion.

Use tools to reload. If AI needs to “recall” compacted content, it will use tools to reread files.

Implications for Building AI Agents

If you want to build your own AI Agent, context management is key:

Set token limits: Don’t assume the model can remember everything—clarify context window size.

Implement compaction mechanism: When approaching limit, identify compressible content and generate summaries.

Preserve key metadata: Even when compacting content, preserve “existence” and “references.”

Let users participate in decisions: For important content, ask users “should this be preserved?”

Monitor context usage: Provide visibility into context usage, help users understand.

Summary

The 200K context window isn’t “unlimited memory,” but a “memory palace” requiring careful management. Through auto compaction, microcompact, token budget strategy, and context collapse, Claude Code packs as much useful information as possible into a limited window.

Understanding this helps you:

• Understand AI’s “forgetting” behavior
• Manage long conversations more effectively
• Draw on these patterns when designing your own AI Agents