Streaming Principles and Event Model: Core Technology for Real-time Output

In a traditional request-response model, an agent must wait until all output has been generated before returning a complete result. That black-box waiting experience is especially poor for long-form generation: users stare at a blank screen for seconds or tens of seconds without knowing whether the backend is working. Streaming turns generation into a fine-grained sequence of events and produces typewriter-style progressive output, reducing perceived latency from total generation time to time to first token.

This article examines ADK Go streaming implementation from three angles: protocol, runtime, and architecture.

Why Streaming Matters

Quantified User Experience Comparison

Metric	Synchronous Mode	Streaming Mode	Improvement
Time to first character	Equal to total generation time	200-500 ms	90%+ reduction
User abandonment rate for waits above 5 seconds	35%	Less than 5%	85% reduction
Peak memory usage	Must cache the full response	Only the current chunk	60-80% reduction
Server-side connection hold time	Short, one-shot return	Long, full lifecycle	Requires dedicated optimization

Performance Insight: According to Google Research, keeping TTFT under 300 ms raises user satisfaction with AI-generated content by 47%. The value of streaming is not that it makes generation faster; it lowers psychological waiting time through progressive presentation.

Technical Principles: HTTP/2 and SSE

HTTP/2 Multiplexing Basics

ADK Go streaming depends on HTTP/2 as the transport layer. Compared with HTTP/1.1, HTTP/2 provides key capabilities for SSE:

HTTP/1.1 limits:
+---------------------------------------------------+
| Connection 1: Request A -------------------------|  Head-of-line blocking
|            Request B waits for A                 |  6-8 concurrent connections per host
+---------------------------------------------------+

HTTP/2 multiplexing:
+---------------------------------------------------+
| Stream 1: Request A ████░░░░░░░░░░░░░░░░░░      |  Multiple streams in one connection
| Stream 3: Request B ░░████░░░░░░░░░░░░░░░░░░      |  Stream priority and flow control
| Stream 5: SSE stream ░░░░░░███████████████████   |  Long-lived connection, server push support
+---------------------------------------------------+

Binary framing: HTTP/2 splits all communication into smaller messages and frames encoded in binary form. An SSE stream is an independent stream, usually with an odd ID initiated by the client, and travels in parallel with other requests over a single TCP connection without interfering with them.

Flow control: HTTP/2 window-based flow control lets the receiver announce how much data it is willing to accept. That mechanism becomes the transport-layer foundation for streaming backpressure: when the client cannot process data as fast as the server pushes it, the server can slow down by observing a smaller window.

SSE Protocol Details

SSE is a W3C-standardized server push technology based on a plain-text streaming format:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive

id: 1
event: text_delta
data: {"chunk": "Hello", "index": 0}

id: 2
event: text_delta
data: {"chunk": "world", "index": 1}

id: 3
event: tool_call
data: {"name": "search", "args": {"query": "Go ADK"}}

id: 4
event: done
data: {"finish_reason": "stop", "total_tokens": 42}

Protocol characteristics:

Event format: each message consists of id:, event:, and data: fields, separated by two newline characters. The id field supports the Last-Event-ID reconnection mechanism.
Automatic reconnection: the browser’s native EventSource reconnects automatically after disconnection and sends Last-Event-ID; the server can resume from the event after that ID.
One-way flow: SSE is a server-to-client channel. If the client must send data, such as a cancel request, it must use a separate HTTP request.

Architecture Note: In microservice architectures, SSE’s one-way nature is an advantage. It avoids the state synchronization complexity introduced by bidirectional WebSockets. For agent output, where the server dominates the flow, SSE is lighter weight and easier to scale horizontally than WebSocket.

Connection Lifecycle and Error Recovery

// Production-grade SSE connection manager
type SSEConnection struct {
    ID          string
    Stream      chan Event
    LastEventID int64
    ConnectedAt time.Time
    LastPingAt  time.Time
    mu          sync.RWMutex
    ctx         context.Context
    cancel      context.CancelFunc
}

// Reconnect with exponential backoff
func (c *SSEConnection) ReconnectWithBackoff() {
    backoff := 100 * time.Millisecond
    maxBackoff := 30 * time.Second

    for attempt := 0; attempt < 10; attempt++ {
        err := c.connect()
        if err == nil {
            c.resumeFrom(c.LastEventID)
            return
        }

        time.Sleep(backoff)
        backoff = min(backoff*2, maxBackoff)

        jitter := time.Duration(rand.Int63n(int64(backoff) / 2))
        time.Sleep(jitter)
    }

    c.fallbackToPolling()
}

Buffer Management and Backpressure

Bounded Channels and Backpressure

In streaming, an unbounded buffer can exhaust memory if the server produces events faster than the client consumes them. ADK Go uses bounded channels to implement backpressure:

const StreamBufferSize = 100

type BufferedStream struct {
    events   chan Event     // bounded buffer
    overflow chan Event     // overflow events for monitoring
    dropped  atomic.Int64
}

func NewBufferedStream() *BufferedStream {
    return &BufferedStream{
        events:   make(chan Event, StreamBufferSize),
        overflow: make(chan Event, 10),
    }
}

func (s *BufferedStream) TrySend(event Event) bool {
    select {
    case s.events <- event:
        return true
    default:
        select {
        case s.overflow <- event:
        default:
        }
        s.dropped.Add(1)
        return false
    }
}

This design gives the system three behaviors: fast clients receive events immediately, slow clients trigger non-blocking drops or pauses, and monitoring systems can observe overflow instead of seeing memory silently grow.

Multilevel Buffer Strategy

In production, a single channel buffer is not enough. A complete streaming pipeline usually has four levels of buffering:

Model generation queue
        |
        v
Application event buffer (chan Event)
        |
        v
SSE serialization buffer (bytes.Buffer)
        |
        v
HTTP response buffer (kernel socket buffer)
        |
        v
Client receive buffer

Each layer must be tuned independently:

The model generation queue should be small to limit in-flight tool calls.
The application event buffer should preserve ordering while limiting memory.
The SSE serialization buffer should be flushed immediately after important events.
The HTTP response buffer must avoid being blocked by reverse proxies.
The client receive buffer should be matched to frontend rendering capacity.

func (s *SSEConnection) WriteEvent(event Event) error {
    serialized, err := serializeEvent(event)
    if err != nil {
        return err
    }

    if _, err := s.Writer.Write(serialized); err != nil {
        return err
    }

    if _, err := s.Writer.Write([]byte("\n\n")); err != nil {
        return err
    }

    if flusher, ok := s.Writer.(http.Flusher); ok {
        flusher.Flush()
    }

    return nil
}

Best Practice: Flush after every critical event such as the first token, tool calls, and final completion. For high-frequency text deltas, consider batch flushing after 100 ms or every 10 events to balance latency and throughput.

Connection Lifecycle Management

Complete Lifecycle Model

A production-grade streaming endpoint needs to manage seven lifecycle states explicitly:

Client connection established
        |
        v
SSE handshake completed
        |
        v
First token generated
        |
        v
Steady-state streaming
        |
        v
Client disconnect detected
        |
        v
Graceful resource cleanup

Timeout and Heartbeat

Long-lived SSE connections must include timeouts and heartbeats:

type SSEServer struct {
    idleTimeout     time.Duration
    keepAlivePeriod time.Duration
    maxBufferSize   int
    metrics         *Metrics
}

func (s *SSEServer) serveStream(w http.ResponseWriter, r *http.Request) {
    ctx, cancel := context.WithTimeout(r.Context(), s.idleTimeout)
    defer cancel()

    conn := &SSEConnection{
        ID:          generateID(),
        Stream:      make(chan Event, StreamBufferSize),
        ConnectedAt: time.Now(),
        LastPingAt:  time.Now(),
        ctx:         ctx,
        cancel:      cancel,
    }

    defer conn.Close()

    // Send periodic heartbeats to prevent proxy timeout
    go s.keepAliveLoop(conn)

    if err := s.streamEvents(conn); err != nil {
        s.writeErrorEvent(conn, err)
    }

    conn.writeDoneEvent()
}

func (s *SSEServer) keepAliveLoop(conn *SSEConnection) {
    ticker := time.NewTicker(s.keepAlivePeriod)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            conn.writeComment("keepalive")
            conn.mu.Lock()
            conn.LastPingAt = time.Now()
            conn.mu.Unlock()
        case <-conn.ctx.Done():
            return
        }
    }
}

Graceful Disconnection Handling

When a client leaves, the server must release resources quickly and consistently:

func (s *SSEServer) handleDisconnect(conn *SSEConnection, err error) {
    conn.mu.Lock()
    conn.Cancel()
    conn.mu.Unlock()

    // Release model call resources
    s.releaseModelCall(conn.ID)

    // Release tool call resources
    s.releaseToolCallResources(conn.ID)

    // Record metrics
    duration := time.Since(conn.ConnectedAt)
    s.metrics.RecordDisconnect(conn.ID, err, duration)
}

Production Performance Optimization

Connection Reuse and Resource Pooling

Opening and closing connections for each request wastes CPU and memory. Use connection pools and resource pools:

type SSEPool struct {
    connections *sync.Pool
    writers     *sync.Pool
    serializer  *SerializerPool
}

func NewSSEPool() *SSEPool {
    return &SSEPool{
        connections: &sync.Pool{
            New: func() interface{} {
                return &SSEConnection{
                    Stream: make(chan Event, StreamBufferSize),
                }
            },
        },
        writers: &sync.Pool{
            New: func() interface{} {
                return &SSEWriter{}
            },
        },
    }
}

func (p *SSEPool) GetConnection() *SSEConnection {
    conn := p.connections.Get().(*SSEConnection)
    conn.Reset()
    return conn
}

func (p *SSEPool) PutConnection(conn *SSEConnection) {
    conn.Close()
    p.connections.Put(conn)
}

Asynchronous Serialization

Serialize events asynchronously to avoid blocking the main streaming goroutine:

type AsyncSerializer struct {
    jobs      chan *SerializationJob
    batchSize int
}

type SerializationJob struct {
    Event      Event
    Response   chan []byte
}

func (s *AsyncSerializer) Run() {
    batch := make([]*SerializationJob, 0, s.batchSize)

    for job := range s.jobs {
        batch = append(batch, job)

        if len(batch) >= s.batchSize {
            serialized := s.batchSerialize(batch)
            for i, result := range serialized {
                batch[i].Response <- result
            }
            batch = batch[:0]
        }
    }
}

Zero-Copy Optimization

For high-throughput streaming, avoid unnecessary memory copies:

type ZeroCopyEvent struct {
    Type    EventType
    Data    []byte
    Payload interface{}
}

func (e *ZeroCopyEvent) MarshalBinary() ([]byte, error) {
    buf := make([]byte, 8+len(e.Data))
    binary.LittleEndian.PutUint64(buf[:8], uint64(e.Type))
    copy(buf[8:], e.Data)
    return buf, nil
}

Metrics and Observability

Production streaming systems require detailed observability:

type StreamingMetrics struct {
    ActiveConnections   prometheus.Gauge
    EventsPerSecond     prometheus.Counter
    BytesTransferred    prometheus.Counter
    DroppedEvents       prometheus.Counter
    ReconnectAttempts   prometheus.Counter
    TTFTLatency         prometheus.Histogram
    Throughput          prometheus.Histogram
    ErrorRate           prometheus.Gauge
}

Key monitoring indicators include active connections, events per second, bytes transferred, dropped events, reconnection attempts, TTFT latency, throughput distribution, and error rate.

Common Pitfalls

Proxy Buffering

Reverse proxies such as Nginx often buffer streaming responses by default. This causes severe latency because events wait in the proxy until a full response buffer is ready.

Solution:

location /api/stream {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Connection '';
    chunked_transfer_encoding off;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header X-Accel-Buffering no;
}

Channel Design Mistakes

Common Go channel mistakes in streaming include:

Using unbounded channels, which can exhaust memory.
Sending on a closed channel, which panics.
Forgetting to close channels, which causes goroutine leaks.
Blocking on sends, which creates deadlocks with slow clients.

Timeout Configuration

Incorrect timeout settings are a major source of streaming failures. Recommended settings:

SSE connection idle timeout: 5 to 10 minutes.
Event serialization timeout: 100 ms.
Keep-alive period: 15 to 30 seconds.
Reconnection backoff cap: 30 seconds.

Summary

ADK Go streaming combines protocol design, runtime lifecycle management, and production optimization. Its core engineering value lies in turning one complete response into an ordered, typed event stream.

Mastering streaming means mastering four things: event modeling, backpressure control, connection lifecycle, and production observability. Only then can a system deliver stable real-time output at scale.

The next tutorial will cover streaming event handling and how to build production-ready event consumers.

Frequently Asked Questions

Why do we need streaming?

Streaming splits model generation into a fine-grained sequence of events, enabling typewriter-style progressive output, reducing perceived latency, and improving interaction fluency.

How does streaming compare with synchronous generation?

Token generation speed is effectively the same, but streaming has about 8% to 12% lower total throughput and can improve user-experience metrics by more than 300%; for long text, the end-to-end experience often feels faster.

How should HTTP/2 stream reset be handled?

When a client calls stream.Cancel(), it sends RST_STREAM. The server should stop generation and release resources without affecting other streams on the same connection.

How can streaming be scaled horizontally in large deployments?

Use consistent-hash session affinity to route the same session to the same node, and synchronize session state through shared storage such as Redis.

How do we prevent middle proxies from buffering SSE connections?

Set the response headers X-Accel-Buffering: no and Cache-Control: no-cache to explicitly disable proxy buffering.

Why Streaming Matters#

Quantified User Experience Comparison#

Technical Principles: HTTP/2 and SSE#

HTTP/2 Multiplexing Basics#

SSE Protocol Details#

Connection Lifecycle and Error Recovery#

Buffer Management and Backpressure#

Bounded Channels and Backpressure#

Multilevel Buffer Strategy#

Connection Lifecycle Management#

Complete Lifecycle Model#

Timeout and Heartbeat#

Graceful Disconnection Handling#

Production Performance Optimization#

Connection Reuse and Resource Pooling#

Asynchronous Serialization#

Zero-Copy Optimization#

Metrics and Observability#

Common Pitfalls#

Proxy Buffering#

Channel Design Mistakes#

Timeout Configuration#

Summary#