梦兽编程™AI_SUITE After many rounds of conversation, the context grows longer and token costs keep rising. Context Caching caches the unchanging parts of the context and only sends the changed parts, saving money.
The Token Cost Problem
Round 1: 100 tokens
Round 2: 200 tokens (includes context from round 1)
Round 3: 300 tokens (includes context from rounds 1–2)
...
Round 50: 5,000 tokens
The more rounds, the more tokens, the higher the cost. Also, Gemini’s context window is limited; exceeding it causes truncation.
How Context Caching Works
The conversation is split into two parts:
┌─────────────────────────┐
│ Cached Context │ ← Unchanged, cached
│ (system prompt, │
│ tool definitions) │
├─────────────────────────┤
│ Current Context │ ← Changes each round
│ (current turn dialog) │
└─────────────────────────┘
The model only needs to process Current Context; Cached Context is not transmitted repeatedly.
Usage
1. Configure Caching
cache := contextcache.NewLRUCache(1000) // Cache up to 1000 items
session, _ := sessions.NewSession(ctx,
sessions.WithUserID("user-123"),
sessions.WithAgentID("my-agent"),
sessions.WithContextCache(cache),
)
2. Define Cached Content
session.SetCachedContext(ctx, []string{
"You are a professional technical assistant.",
"The following is the list of available tools: ...",
"Reference documentation: ...",
})
3. Automatic Effect
On subsequent requests, cached content is not sent repeatedly, but the model “appears” to have the full context.
Effect Comparison
| Scenario | No Cache | With Cache | Savings |
|---|---|---|---|
| 50-round conversation | 5,000 tokens/round | 200 tokens/round | ~96% |
| Very long document analysis | 8,000 tokens/round | 500 tokens/round | ~94% |
Notes
Caching is not suitable for frequently changing contexts
If the context changes every round (lots of new user input), caching has limited effect. Suitable scenarios:
- System prompt
- Tool definitions
- Knowledge base documents
- Reference FAQ
Cache has size limits
Each cache entry has a size limit. Content that is too long cannot be cached and needs compression or truncation.
FAQ
Q: Does cached content expire?
A: You can set an expiration time. session.SetCachedContext(ctx, content, cache.WithTTL(time.Hour))
Q: When to cache and when to send full context? A: The framework handles it automatically. But note that the first request and when cached content changes require sending the full context to establish the cache.
Q: Does the cache use memory?
A: LRUCache automatically cleans up the least recently used cache entries when memory is full.
Next Steps
Caching stores context, Compression compresses context — combining both makes long conversation costs manageable.
← Event System | Context Compression →
Want to learn more Go ADK hands-on? Follow the 「全栈之巅-梦兽编程」 (Full Stack Summit - Dream Beast Programming) WeChat official account for weekly Go / AI programming tips.
