Microcompact - The Art of Precise Context Pruning

Ever had this happen? When moving to a new place, you seal up entire boxes of stuff to carry over (auto-compaction), but what about those scattered little items in your drawers? Carrying entire boxes is too bulky; throwing them away is risky because there might be something useful inside. What you need is “precise organization” - keep what matters, toss what doesn’t.

That’s exactly what Microcompact does.

Auto-Compaction is Moving, Microcompact is Organizing Drawers

In the previous article, we discussed auto-compaction - when the house gets full, pack up the entire room into a summary. This is effective but somewhat crude, like sealing your entire bookshelf into boxes and having to unpack everything just to find one book.

Microcompact is different - it’s “precise pruning”:

To use an analogy:

• Auto-compaction: Packing up the entire study when moving
• Microcompact: Throwing used scratch paper into the trash while organizing your desk

Working together, AI memory management becomes both efficient and precise.

Three “Cleaners” in Microcompact

Claude Code actually has three “cleaners” taking shifts, each responsible for different scenarios:

Cleaner #1: Time-Triggered (Post-Lunch Cleanup)

Imagine: you used Claude Code to look up a bunch of information in the morning, then went to lunch. You come back at 1 PM to continue - three hours have passed in between.

What happens then? The server cache has expired. Anthropic’s cache validity is 1 hour, so after 3 hours it’s definitely expired. This means when you ask your next question, the server has to recache your entire conversation history - costing quite a bit.

The time-triggered microcompact logic is straightforward: Since the cache has already expired, might as well clear the old content first, making the rewritten content smaller and cheaper.

What does it do?

1. Check how long ago the “last AI response” was (default: over 60 minutes)
2. Collect all compressible tool results (searches, file reads, etc.)
3. Keep only the 5 most recent ones, replacing the rest with placeholders: [Old tool result content cleared]
4. When the next conversation happens, the rewritten content is smaller, saving money

It’s like coming back from lunch to find the janitor has already cleared your scratch paper from the desk - since you weren’t there anyway, keeping it served no purpose.

Cleaner #2: Cache Microcompaction (Precision Surgery That Doesn’t Break Cache)

Time-triggered has a problem: it modifies message content, which changes the cache prefix, requiring recaching on the next request.

But in real-time conversations, this is unacceptable - the cache you’ve been building up could be worth tens of thousands of tokens, you can’t just throw it away.

Cache microcompaction uses a clever approach: Instead of modifying local messages, it tells the server “delete this content from the cache”.

This is called cache_edits, working like remote control:

AI: Hello server, please delete tool results #3, #5, #7 from the cache
Server: Got it, deleted. Cache prefix is still continuous, can still hit on next request
AI: Received, continuing conversation

This way, local messages are untouched, server cache isn’t broken, but content is indeed cleaned up - best of both worlds.

When does this cleaner work?

• When accumulated compressible tool results exceed a threshold (e.g., 20)
• Keep the most recent few, mark the rest as “pending deletion”
• On the next API call, use cache_edits command to have the server delete them

Cleaner #3: API Context Management (Declarative Butler)

The first two are “imperative” - the program decides what to delete and when. API Context Management is “declarative”: you tell the server “when exceeding 180k tokens, help me clean up”, and the server executes on its own.

You can configure it like this:

• Trigger when input tokens exceed 180,000
• Keep the most recent 40,000 tokens
• Clean up results from these tools: Bash, Grep, Glob, FileRead, etc.
• But keep FileEdit and FileWrite instructions

It’s like hiring a smart butler: no need to give orders every time, they clean automatically according to your set rules.

Priority of the three mechanisms:

1. Time-triggered checked first (if cache expired, no point checking others)
2. Cache microcompaction second (cache still hot, precise cleanup)
3. API Context Management always present (declarative baseline)

What Gets “Thrown Away”?

Not everything gets cleaned up by microcompaction. The three cleaners have different “cleaning lists”:

Time-triggered and cache microcompaction clean:

• Read: File read results
• Bash: Command output
• Grep: Search results
• Glob: File match lists
• WebSearch, WebFetch: Web content
• Edit, Write: File edit/write results

API Context Management additionally handles:

• NotebookEdit: Notebook edits
• FileEdit, FileWrite instructions (not just results)
• thinking blocks: Extended thinking model’s thought process

What won’t be cleaned:

• Conversation messages themselves (user and AI chat)
• System prompts
• CLAUDE.md content
• Tool call instructions (cleaning results doesn’t clean the calls themselves)

It’s like the janitor only takes out the trash, doesn’t touch the books on your desk.

Elegant Design Details in Microcompact

The microcompact system has several engineering gems worth learning from:

1. Cache Interruption Coordination

There’s a dedicated module monitoring “whether cache was interrupted” - if cache hit tokens suddenly drop, it raises an alarm. But microcompact deliberately reduced cache content, which shouldn’t count as “interruption”.

How was this solved?

• Before executing microcompaction, notify the detector: “I’m about to delete stuff, don’t panic”
• The detector sees the flag and knows this drop is expected, doesn’t report error

It’s like telling the security guard: “I’m taking out the trash, don’t mistake me for a thief.”

2. Subagent Isolation

Claude Code can fork subagents (e.g., agents dedicated to searching). Each subagent has its own conversation history.

Microcompaction only executes on the main thread. Why?

If a subagent’s tool results were registered in global state, the main thread might say “I’m deleting tool #7” on its next execution - but this tool doesn’t exist in the main thread’s messages, causing invalid operations.

By checking isMainThreadSource(), subagents are excluded, each cleaning their own rooms.

3. Idempotency Guarantee

Time-triggered could be executed multiple times (though it shouldn’t). The code has guards:

if (block.content !== TIME_BASED_MC_CLEARED_MESSAGE) {
  // Clean up and count
}

If already cleaned, don’t count token savings again - avoiding double calculation.

It’s like only recording an expense once in your accounting.

Practical: Understanding AI’s “Selective Forgetting”

Understanding microcompaction helps you use Claude Code better:

Scenario 1: AI Says “I Don’t Remember That Search Result”

After a long conversation, AI might forget a previous grep result. This isn’t hallucination - microcompaction cleared the old result.

Cleared results look like this:

[Old tool result content cleared]

If you need to reference it again, just let AI search again - more reliable than having it “recall” cleared content.

Scenario 2: Back After an Hour, AI “Forgot”

If you’ve been away over 1 hour, time-triggered may have already cleared most old tool results (keeping only the 5 most recent). This is normal - because the cache expired, clearing old content reduces recaching costs.

When you return, having AI re-read key files is normal operation.

Scenario 3: Where to Put Important Info to Keep It Safe?

Microcompaction only clears tool results, doesn’t affect system prompts. So:

• CLAUDE.md: Safe, won’t be cleared
• Tool results: May be cleared
• Conversation history: Safe, but will be compacted

Put key project info in CLAUDE.md, like writing on a whiteboard - the janitor won’t wipe it.

Scenario 4: Some Results “Lost” During Parallel Searches

When AI makes multiple searches simultaneously, if total results exceed 200K character limit, some results are persisted to disk (prompting “Output too large, saved to file”).

This is the budget mechanism preventing context inflation. You can reduce per-search output with more precise search conditions.

Microcompact vs Auto-Compaction: A Tag Team

The two mechanisms work together forming a complete context management strategy:

It’s like managing a room:

• Regular tidying (microcompact)
• Weekend deep clean (auto-compaction)
• Moving and decluttering (truncation)

Implications for Building AI Agents

If you want to design your own AI agent, the microcompact mechanism is worth borrowing from:

1. Layered Cleanup Strategy

Don’t just have one “compress” button - layer it:

• Fine-grained: Individual content cleanup (microcompact)
• Medium-grained: Batch cleanup of related content
• Coarse-grained: Overall summarization (auto-compaction)

2. Distinguish “Cleanable” vs “Non-cleanable”

Clearly define which content can be discarded:

• Search results: Cleanable (can search again)
• File content: Cleanable (can read again)
• User instructions: Non-cleanable (lost means gone)
• System config: Non-cleanable (affects behavior)

3. Leverage External Capabilities

Anthropic’s cache_edits lets clients remotely manipulate server-side cache. If your API supports similar capabilities, you can achieve “cleanup without breaking prefix”.

4. Expectation Management

Users need to know “why did AI forget.” Display in UI:

• Which tool results were cleared
• Why they were cleared (time/quantity/size)
• How to retrieve again (re-execute tool)

Summary

Microcompact is the “fine scalpel” of Claude Code’s context management:

• Three mechanisms: Time-triggered (after cache expiry), cache microcompaction (precise surgery), API Context Management (declarative butler)
• Layered collaboration: Microcompact handles details, auto-compaction handles the big picture
• Smart tradeoffs: Only cleans rebuildable content, preserves critical information
• User awareness: Understanding “AI forgot” may be due to microcompaction, not a bug

It’s like organizing a room:

• Auto-compaction is seasonal deep cleaning
• Microcompact is daily tidying
• Together, they make a house comfortable to live in

Understanding microcompact lets you:

• Understand AI’s “selective forgetting” in long conversations
• Wisely choose where to store important information
• Implement similar精细 management in your own AI agents

Next up: Token Budget - How to Spend 200K Wisely.