Agent Runtime Architecture: How ADK Go Executes Agents

Getting an Agent to run and getting it to run well are two different problems. In production, an Agent system’s stability, performance, and observability largely depend on the quality of its underlying Runtime. ADK Go’s Runtime is not a thin HTTP wrapper — it is a carefully designed concurrent execution framework that must strike a fine balance between high throughput, long-lived connections, state retention, and memory safety.

This post dissects ADK Go’s Runtime architecture at the source level, covering its process model, concurrency scheduling strategy, memory management, and the pitfalls and solutions you will meet in real production.

Runtime vs ADK Web: The Essential Difference

Before diving into the technical details, two concepts must be disentangled:

Component	Purpose	Production-ready	Design goal
ADK Web	Development and debugging	❌	Validate Agent logic quickly; single-user interaction
Agent Runtime	Production deployment	✅	High concurrency, high availability, observability, scalability

ADK Web is fundamentally a development aid. It usually runs as a single process and lacks production features such as connection pool management, request rate limiting, and health checks. Its design philosophy is “fast to start, instant feedback,” which is ideal for iteration in development.

Agent Runtime, by contrast, is a long-running service process that must handle a set of production-grade challenges:

Concurrency safety: serving hundreds or thousands of users at once, each request potentially triggering many rounds of LLM calls
State isolation: sessions between users must be strictly isolated to prevent data leakage
Resource control: prevent a single complex request from exhausting the system (memory, CPU, connections)
Graceful shutdown: on deploy, wait for in-flight requests to complete instead of killing them
Observability: expose metrics, tracing, and structured logging for operations

Understanding this difference is the first step toward avoiding the classic “works in dev, breaks in prod” failure.

Process Model: Single-Process Multi-Goroutine Architecture

ADK Go Runtime uses the classic single-process multi-goroutine model. This pattern is ubiquitous in the Go ecosystem but has been tuned specifically for the Agent use case:

┌─────────────────────────────────────────────────────────────┐
│                    Agent Runtime Process                      │
│                                                             │
│  ┌──────────────────┐    ┌──────────────────────────────┐  │
│  │   HTTP Server    │───→│      Router / Dispatcher     │  │
│  │   (net/http)     │    │   (request routing & fan-out)│  │
│  └──────────────────┘    └──────────────────────────────┘  │
│            │                          │                     │
│            ▼                          ▼                     │
│  ┌──────────────────┐    ┌──────────────────────────────┐  │
│  │  Agent Executor  │    │      Session Manager         │  │
│  │  (executor pool) │    │   (state & persistence)      │  │
│  │                  │    │                              │  │
│  │  ┌────────────┐  │    │  ┌────────┐  ┌──────────┐   │  │
│  │  │ goroutine  │  │    │  │ Active │  │ Expired  │   │  │
│  │  │    #1      │  │    │  │Session │  │ Session  │   │  │
│  │  ├────────────┤  │    │  │  Map   │  │  Queue   │   │  │
│  │  │ goroutine  │  │    │  └────────┘  └──────────┘   │  │
│  │  │    #2      │  │    │                              │  │
│  │  ├────────────┤  │    │  ┌──────────────────────┐    │  │
│  │  │    ...     │  │    │  │   Persistence        │    │  │
│  │  ├────────────┤  │    │  │ (Redis / PostgreSQL) │    │  │
│  │  │ goroutine  │  │    │  └──────────────────────┘    │  │
│  │  │    #N      │  │    │                              │  │
│  │  └────────────┘  │    └──────────────────────────────┘  │
│  └──────────────────┘                                       │
│                                                             │
│  ┌──────────────────┐    ┌──────────────────────────────┐  │
│  │  Metrics Server  │    │  Graceful Shutdown Handler   │  │
│  │   (Prometheus)   │    │  (shutdown & resource reclamation) │
│  └──────────────────┘    └──────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Why Choose a Single-Process Model

In the Agent domain, single-process multi-goroutine beats multi-process or thread-based models in several ways:

Memory sharing: Session data can be shared across goroutines without the serialization cost of cross-process IPC.
Lightweight scheduling: Go’s GMP scheduler (Goroutine–Machine–Processor) is far cheaper than OS-thread scheduling.
Fast context switching: a goroutine switch costs ~2 KB of stack, which is ideal for the frequent I/O waits in Agent workloads (LLM API calls).

But a single process also has limits:

CPU ceiling: bound by GOMAXPROCS in one process; cannot scale across machines.
Weak fault isolation: a single unrecovered panic can bring down the whole process.
Memory ceiling: 4 GB on 32-bit systems; on 64-bit, large heaps trigger GC pause pressure.

Deep Dive into Core Components

HTTP Server: More Than Listening on a Port

The Runtime’s HTTP layer carries serious production hardening:

// Production HTTP Server configuration
runtime, err := agentruntime.New(ctx,
    agentruntime.WithPort(8080),
    agentruntime.WithAgent(agent),
    agentruntime.WithReadTimeout(30 * time.Second),
    agentruntime.WithWriteTimeout(60 * time.Second),
    agentruntime.WithMaxHeaderBytes(1<<20),
    agentruntime.WithIdleTimeout(120 * time.Second),
)
if err != nil {
    log.Fatalf("failed to create runtime: %v", err)
}

server := &http.Server{
    Addr:           ":8080",
    Handler:        runtime.Handler(),
    ReadTimeout:    30 * time.Second,
    WriteTimeout:   60 * time.Second,
    IdleTimeout:    120 * time.Second,
    MaxHeaderBytes: 1 << 20,
    TLSConfig: &tls.Config{
        MinVersion: tls.VersionTLS12,
        CurvePreferences: []tls.CurveID{
            tls.X25519,
            tls.CurveP256,
        },
        PreferServerCipherSuites: true,
        CipherSuites: []uint16{
            tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
            tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
            tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
            tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
        },
    },
}

Production experience:

WriteTimeout must be long enough (60s+) because an Agent may chain several LLM calls with network latency at each hop.
IdleTimeout at 120s is a good balance between connection reuse and resource reclamation.
Always enable TLS 1.2+ and disable weak cipher suites.

Agent Executor: The Concurrency Core

The Executor is the heart of the Runtime — it decides how Agent requests are handled concurrently. ADK Go’s Executor uses a Worker Pool + Unbounded Queue hybrid:

type Executor struct {
    agent         *llmagent.Agent
    maxConcurrent int
    queueSize     int
    sem           chan struct{}
    queue         chan *Request
}

func (e *Executor) Execute(ctx context.Context, req *Request) (*Response, error) {
    select {
    case e.sem <- struct{}{}:
        defer func() { <-e.sem }()
        return e.executeWithTimeout(ctx, req)
    default:
        select {
        case e.queue <- req:
            return e.waitForResult(ctx, req)
        case <-ctx.Done():
            return nil, ctx.Err()
        default:
            return nil, fmt.Errorf("server overloaded: queue full (size=%d)", e.queueSize)
        }
    }
}

func (e *Executor) executeWithTimeout(ctx context.Context, req *Request) (*Response, error) {
    ctx, cancel := context.WithTimeout(ctx, 5*time.Minute)
    defer cancel()

    resultCh := make(chan *Response, 1)
    errCh := make(chan error, 1)

    go func() {
        defer func() {
            if r := recover(); r != nil {
                errCh <- fmt.Errorf("panic recovered: %v\n%s", r, debug.Stack())
            }
        }()

        result, err := e.agent.Run(ctx, req.Input)
        if err != nil {
            errCh <- err
            return
        }
        resultCh <- result
    }()

    select {
    case result := <-resultCh:
        return result, nil
    case err := <-errCh:
        return nil, err
    case <-ctx.Done():
        return nil, fmt.Errorf("request timeout: %w", ctx.Err())
    }
}

Key design decisions:

Semaphore over fixed worker pool: Agent request durations vary wildly (simple queries ~1s; complex reasoning 60s+). A fixed pool lets slow requests starve fast ones; a semaphore-based approach is more flexible.
One goroutine per request: goroutines are cheap, but a flood of concurrent goroutines raises GC pressure — pair with maxConcurrent limits in production.
Panic recovery is mandatory: LLM output can be malformed and Tool implementations can panic; recover protects the process.

Session Manager: The Art of State Management

The Session Manager is among the Runtime’s most complex components. It handles state isolation, expiry, and persistence:

runtime, err := agentruntime.New(ctx,
    agentruntime.WithMaxSessions(10000),
    agentruntime.WithSessionTTL(time.Hour*24),
    agentruntime.WithSessionCheckInterval(time.Minute*5),
    agentruntime.WithSessionStore(redisStore),
)

type Session struct {
    ID         string
    UserID     string
    AgentID    string
    State      map[string]interface{}
    History    []Message
    CreatedAt  time.Time
    LastAccess time.Time
    mu         sync.RWMutex
}

Per-Session memory estimate:

Item	Size	Note
Session metadata	~500 B	ID, timestamps, config
Conversation history (100 turns)	~50 KB	~500 chars per turn
Agent state	~10 KB	depends on tool payloads
Total	~60 KB / session

By that math, 10 000 sessions consume ~600 MB. If histories run long (1000+ turns) or carry large file blobs, memory can spike to several GB.

Production recommendations:

Set a hard MaxSessions cap to prevent OOM.
Set a sensible TTL; 24h is a good default since most users do not resume the same conversation the next day.
Truncate or summarize long histories — keep only the last N turns.
Use Redis or similar for cross-instance session sharing.

Concurrency Model: From Theory to Practice

Goroutine Count vs System Resources

Concurrent requests	Goroutines	Memory (approx)	Note
1	1 + background	~10 MB	Baseline
100	100 + background	~50 MB	Normal load
1 000	1 000 + background	~200 MB	High concurrency, tune GC
10 000	10 000 + background	~1 GB+	Very high, scale horizontally

Note: these numbers are only for the Executor. Add HTTP connection-handling goroutines, background cleanup goroutines, and metrics collection goroutines for the real total.

Sizing the Concurrency Cap

func calculateMaxConcurrent() int {
    numCPU := runtime.GOMAXPROCS(0)

    var m runtime.MemStats
    runtime.ReadMemStats(&m)
    availableMem := m.Sys - m.HeapAlloc

    // assume ~10 MB per concurrent request (stack + session data)
    memBased := int(availableMem / (10 * 1024 * 1024))

    // CPU bound: 10 concurrent per core, accounting for I/O wait
    cpuBased := numCPU * 10

    // Take the smaller and leave a 20% headroom
    maxConcurrent := int(float64(min(memBased, cpuBased)) * 0.8)

    if maxConcurrent < 10 {
        maxConcurrent = 10
    }
    return maxConcurrent
}

runtime, err := agentruntime.New(ctx,
    agentruntime.WithMaxConcurrent(calculateMaxConcurrent()),
    agentruntime.WithQueueSize(calculateMaxConcurrent() * 5),
)

Memory Management: Practical OOM Avoidance

Session Memory Optimization

// 1. History truncation — keep only the last 20 turns
type TruncatedHistory struct {
    maxRounds int
    messages  []Message
}

func (h *TruncatedHistory) Add(msg Message) {
    h.messages = append(h.messages, msg)
    if len(h.messages) > h.maxRounds {
        h.messages = h.messages[len(h.messages)-h.maxRounds:]
    }
}

// 2. History summarization — LLM-summarize old history
func (s *Session) SummarizeHistory(ctx context.Context) error {
    if len(s.History) < 50 {
        return nil
    }

    oldMessages := s.History[:len(s.History)-20]
    recentMessages := s.History[len(s.History)-20:]

    summary, err := s.summarize(ctx, oldMessages)
    if err != nil {
        return err
    }

    s.History = append([]Message{{Role: "system", Content: summary}}, recentMessages...)
    return nil
}

// 3. Large-object offloading — push big blobs to object storage
func (s *Session) StoreLargeContent(content []byte) (string, error) {
    if len(content) > 1024*1024 {
        key := fmt.Sprintf("session/%s/%d", s.ID, time.Now().Unix())
        if err := s.objectStore.Put(key, content); err != nil {
            return "", err
        }
        return key, nil
    }
    return "", nil
}

Automatic Cleanup

func (sm *SessionManager) startCleanup(ctx context.Context) {
    ticker := time.NewTicker(sm.checkInterval)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            sm.cleanupExpired()
        case <-ctx.Done():
            return
        }
    }
}

func (sm *SessionManager) cleanupExpired() {
    now := time.Now()
    var expiredCount int

    sm.sessions.Range(func(key, value interface{}) bool {
        session := value.(*Session)

        session.mu.RLock()
        isExpired := now.Sub(session.LastAccess) > sm.ttl
        session.mu.RUnlock()

        if isExpired {
            if sm.store != nil {
                if err := sm.store.Save(session); err != nil {
                    log.Printf("failed to persist session %s: %v", session.ID, err)
                }
            }
            sm.sessions.Delete(key)
            expiredCount++
        }
        return true
    })

    if expiredCount > 0 {
        log.Printf("cleaned up %d expired sessions", expiredCount)
        runtime.GC()
    }
}

Common Production Issues and Fixes

Q: What if the Runtime process crashes?

Root causes:

OOM kill — the OS terminates the process for exceeding memory limits.
Unrecovered panic — a single goroutine’s panic takes down the process.
Deadlock — incorrect concurrency control blocks all workers.
Resource leak — leaked goroutines or connections exhaust the process.

Solutions:

// 1. Process management — systemd auto-restart
// /etc/systemd/system/my-agent.service
//
// [Unit]
// Description=ADK Go Agent Runtime
// After=network.target
//
// [Service]
// Type=notify
// ExecStart=/usr/local/bin/my-agent
// Restart=always
// RestartSec=5
// StartLimitIntervalSec=300
// StartLimitBurst=3
// MemoryMax=2G
// MemorySwapMax=0
// TasksMax=10000
// TimeoutStopSec=30
// KillSignal=SIGTERM
//
// [Install]
// WantedBy=multi-user.target

// 2. Code-level panic recovery
func safeRun(ctx context.Context, agent *llmagent.Agent, input string) (result *Response, err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("panic recovered: %v\nstack: %s", r, debug.Stack())
            metrics.PanicCounter.Inc()
        }
    }()

    return agent.Run(ctx, input)
}

// 3. Health check endpoint
http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
    if err := checkDependencies(); err != nil {
        w.WriteHeader(http.StatusServiceUnavailable)
        _ = json.NewEncoder(w).Encode(map[string]string{"status": "unhealthy", "reason": err.Error()})
        return
    }
    _ = json.NewEncoder(w).Encode(map[string]string{"status": "healthy"})
})

Q: Sessions are eating all the memory

Diagnosis:

Enable pprof with import _ "net/http/pprof" and inspect /debug/pprof/heap.
Read the heap profile to confirm the share held by Session objects.
Monitor the Session growth curve — a leak will keep rising instead of settling.

Fixes:

Lower MaxSessions to a value computed from available memory.
Shorten SessionTTL (e.g., 4h instead of 24h).
Enable history truncation or summarization.
Move cold data to Redis; keep only hot data in memory.

Q: Request volume is overwhelming the Runtime

Horizontal scaling topology:

                    ┌─────────────┐
                    │  Nginx / ALB │
                    │(Load balancer)│
                    └──────┬──────┘
                           │
           ┌───────────────┼───────────────┐
           ▼               ▼               ▼
      ┌─────────┐     ┌─────────┐     ┌─────────┐
      │Runtime  │     │Runtime  │     │Runtime  │
      │ Instance 1 │  │ Instance 2 │  │ Instance 3 │
      └────┬────┘     └────┬────┘     └────┬────┘
           │               │               │
           └───────────────┼───────────────┘
                           ▼
                    ┌─────────────┐
                    │    Redis     │
                    │ (shared session storage)
                    └─────────────┘

Key configurations:

Load balancing via Round Robin or Least Connections.
Sessions must live in a shared store such as Redis.
Consider Sticky Sessions to minimize cross-instance state sync.

Production Tuning Checklist

Before deploying to production, verify each item:

Concurrency cap: MaxConcurrent sized from CPU and memory.
Timeouts: ReadTimeout, WriteTimeout, IdleTimeout configured.
Session limits: MaxSessions and SessionTTL set.
Panic recovery: every goroutine entry uses defer recover.
Resource limits: systemd or Docker enforces CPU and memory caps.
Health check: /health endpoint implemented and probes critical dependencies.
Graceful shutdown: SIGTERM support; wait for in-flight requests to finish.
Log level: production uses info or warn; disable debug.
Monitoring: Prometheus metrics and tracing configured.
TLS: TLS 1.2+ with weak cipher suites disabled.

Next Steps

With the Runtime architecture understood, the next step is learning how to deploy it in production — starting with the CLI.

← Streaming UI | CLI Deployment →

Want to learn more Go ADK hands-on? Follow the “Mengshou Programming” channel for weekly updates on Go and AI programming.

Runtime vs ADK Web: The Essential Difference#

Process Model: Single-Process Multi-Goroutine Architecture#

Why Choose a Single-Process Model#

Deep Dive into Core Components#

HTTP Server: More Than Listening on a Port#

Agent Executor: The Concurrency Core#

Session Manager: The Art of State Management#

Concurrency Model: From Theory to Practice#

Goroutine Count vs System Resources#

Sizing the Concurrency Cap#

Memory Management: Practical OOM Avoidance#

Session Memory Optimization#

Automatic Cleanup#

Common Production Issues and Fixes#

Q: What if the Runtime process crashes?#

Q: Sessions are eating all the memory#

Q: Request volume is overwhelming the Runtime#

Production Tuning Checklist#

Next Steps#

Frequently Asked Questions