Streaming UI and Reconnection Pattern

The Streaming UI and Reconnection Pattern governs how interactive web and mobile frontends consume incremental, long-lived server streams—such as LLM token generation, multi-step AI agent progress timelines, real-time workflow status updates, and live collaboration feeds—without losing frames, duplicating rendered messages, leaking proxy buffers, or slamming servers with thundering-herd reconnect storms when networks drop.

Real-world client networks are hostile: corporate VPNs terminate idle sockets at 60 seconds, mobile devices switch between Wi-Fi and LTE mid-stream, browser tabs sleep, and load balancers recycle upstream pods during rolling deploys. Naive fetch() readers or bare new EventSource() setups silently hang forever on half-open TCP links or drop half an AI response on reconnect. For transport and frontend foundations, consult Frontend Development Hub, Web Services and APIs Hub, Server-Sent Events Patterns, and WebSocket Patterns.


1. Transport Selection and Checkpointed Stream Architecture

Select the simplest transport matching the communication symmetry (API Protocol Comparison):

sequenceDiagram
    participant UI as Browser Client (Stream Store)
    participant BFF as Edge BFF (No Buffering)
    participant Buf as Stream Buffer (Redis Streams / DB Log)
    participant Worker as Producer / Agent Worker

    UI->>BFF: POST /streams/runs/42/attach (Last-Event-Seq: 0)
    BFF->>Buf: Subscribe from seq > 0
    Worker->>Buf: Append {seq: 1, token: "Analyzing"}
    Buf-->>BFF: Frame {seq: 1}
    BFF-->>UI: SSE id: 1 / data: {...}
    Note over UI,BFF: Wi-Fi drops before seq 4 arrives!
    Worker->>Buf: Append {seq: 2..5} (Buffered with TTL)
    UI->>BFF: Reconnect after jittered backoff (Last-Event-Seq: 1)
    BFF->>Buf: Replay missed frames seq > 1 (yields 2..5)
    BFF-->>UI: SSE id: 2..5 (UI deduplicates seq <= 1)
Streaming MechanismDirectional ModelCustom Auth Headers / POST BodyResumption PrimitivesPrimary Proxy Failure Mode
Native EventSource (GET)Server -> ClientNo custom headers (relies on cookies/URL params); GET onlyBuilt-in Last-Event-ID on browser auto-reconnectSilent 60 s idle close by ALB/Nginx if no heartbeat comment (: keepalive)
fetch + ReadableStream SSEServer -> Client (POST start allowed)Full POST JSON payloads, CSRF/custom headers, AbortController cancelManual lastEventId tracking + jittered retry loopReverse-proxy response buffering (X-Accel-Buffering not disabled; gzip buffering)
WebSockets (wss://)Full-duplex bidirectionalAuth handshake during upgrade or first auth frameApplication-defined resumeToken + lastAckSeqStateful pod scaling drops connections without pub/sub fanout + ping/pong

Notice a critical ergonomic detail for AI chat and agentic tools: starting a stream almost always requires a POST request with structured JSON inputs (prompt, attachments, idempotencyKey), which native new EventSource(url) cannot issue directly. Therefore, enterprise applications routinely pair POST /runs (returns runId + persists initial run record) followed by resumable GET /runs/:runId/events?afterSeq=N—allowing native EventSource or resilient fetch SSE clients to reconnect cleanly to an already-running background execution without ever re-triggering the LLM prompt or billing twice!


2. Reconnection Storms, Buffering Math, and Jitter Backoff

When an ingress gateway restarts or a brief cellular outage disconnects N = 15{,}000 active streaming clients simultaneously, synchronized naive retries every 1{,}000\text{ ms} create catastrophic spike waves that keep the BFF pinned in an overload cycle (Circuit Breaker Pattern).

Using Exponential Backoff with Full Jitter spreads reconnections uniformly across an expanding window:

t_{\text{delay}}(k) = \text{Uniform}\!\left(0,\; \min\!\left(T_{\max},\; T_{\text{base}} \cdot 2^{k}\right)\right)

where k = 0, 1, 2, \dots is the consecutive failure count, T_{\text{base}} = 500\text{ ms}, and T_{\max} = 30{,}000\text{ ms}. On the first drop (k = 0), clients scatter evenly over [0, 500]\text{ ms}; by k = 3, retries spread over [0, 4{,}000]\text{ ms}:

Equally important is sizing the producer replay ring buffer (Caching Strategies, Redis Streams XADD ... MAXLEN ~ or DB event table). Suppose an LLM emits R = 45\text{ tokens/sec} across an average T_{\text{run}} = 40\text{ s} reasoning stream, where each SSE event envelope averages 220\text{ bytes}:

M_{\text{run}} = R \times T_{\text{run}} \times B_{\text{frame}} = 45 \times 40 \times 220\text{ B} = 396{,}000\text{ B} \approx 387\text{ KB/run}

For 1{,}000 concurrent streaming users, retaining the entire replay buffer for a T_{\text{ttl}} = 15\text{ minute} reconnection grace window costs only \approx 387\text{ MB} of Redis RAM—cheap insurance compared to aborting an expensive 30-second agent run or losing partial output when a user's laptop sleeps for 45 seconds!


3. Resumable State Reconciliation and Proxy/BFF Streaming Contract

A resilient streaming client maintains an explicit connection state machine (IDLE -> CONNECTING -> STREAMING -> RECONNECTING -> PAUSED_OFFLINE -> TERMINAL) inside a single deterministic store (State Management Patterns, Single-Page Application Architecture) rather than scattered useEffect hooks inside React components (React Best Practices).

Client-Side Resilient Stream Consumer (Typed Events + Deduplication)

// Resilient SSE stream consumer with sequence deduplication, heartbeat watchdog, & full jitter
export interface StreamFrame<T = unknown> {
  readonly seq: number;
  readonly eventType: "delta" | "step_status" | "approval_required" | "done" | "error";
  readonly payload: T;
}

export type StreamConnectionPhase =
  | "IDLE"
  | "CONNECTING"
  | "STREAMING"
  | "RECONNECTING"
  | "COMPLETED"
  | "FAILED";

export interface StreamConsumerOptions<T> {
  readonly url: string;
  readonly initialLastSeq?: number;
  readonly heartbeatTimeoutMs?: number; // fail fast on silent half-open TCP drop
  readonly maxRetries?: number;
  readonly onPhaseChange: (phase: StreamConnectionPhase) => void;
  readonly onFrame: (frame: StreamFrame<T>) => void;
}

export async function consumeResumableSseStream<T>(
  opts: StreamConsumerOptions<T>,
  signal: AbortSignal
): Promise<void> {
  let lastAppliedSeq = opts.initialLastSeq ?? 0;
  let attempt = 0;
  const maxRetries = opts.maxRetries ?? 8;
  const heartbeatMs = opts.heartbeatTimeoutMs ?? 25_000;

  while (!signal.aborted) {
    opts.onPhaseChange(attempt === 0 ? "CONNECTING" : "RECONNECTING");

    const attemptAbort = new AbortController();
    const onParentAbort = () => attemptAbort.abort();
    signal.addEventListener("abort", onParentAbort, { once: true });

    let watchdog: ReturnType<typeof setTimeout> | undefined;
    const resetWatchdog = () => {
      if (watchdog) clearTimeout(watchdog);
      watchdog = setTimeout(() => attemptAbort.abort(new Error("HEARTBEAT_TIMEOUT")), heartbeatMs);
    };

    try {
      const separator = opts.url.includes("?") ? "&" : "?";
      const targetUrl = `${opts.url}${separator}afterSeq=${lastAppliedSeq}`;
      resetWatchdog();

      const res = await fetch(targetUrl, {
        method: "GET",
        headers: {
          Accept: "text/event-stream",
          "Last-Event-ID": String(lastAppliedSeq),
        },
        signal: attemptAbort.signal,
      });

      if (res.status === 401 || res.status === 403 || res.status === 404) {
        opts.onPhaseChange("FAILED");
        throw new Error(`Non-retryable stream status HTTP ${res.status}`);
      }
      if (!res.ok || !res.body) {
        throw new Error(`Transient stream error HTTP ${res.status}`);
      }

      opts.onPhaseChange("STREAMING");
      attempt = 0; // Reset backoff once valid response stream opens

      const reader = res.body.getReader();
      const decoder = new TextDecoder();
      let buffer = "";

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        resetWatchdog();

        buffer += decoder.decode(value, { stream: true });
        const rawBlocks = buffer.split("\n\n");
        buffer = rawBlocks.pop() ?? "";

        for (const block of rawBlocks) {
          // Heartbeat comments (: keepalive) reset watchdog and require no UI dispatch
          if (!block.trim() || block.startsWith(":")) continue;

          const dataLine = block
            .split("\n")
            .find((line) => line.startsWith("data:"))
            ?.slice(5)
            .trim();
          if (!dataLine) continue;

          const frame = JSON.parse(dataLine) as StreamFrame<T>;
          // Idempotent sequence deduplication prevents duplicate tokens on replay overlap
          if (frame.seq <= lastAppliedSeq) {
            continue;
          }
          lastAppliedSeq = frame.seq;
          opts.onFrame(frame);

          if (frame.eventType === "done" || frame.eventType === "error") {
            opts.onPhaseChange(frame.eventType === "done" ? "COMPLETED" : "FAILED");
            return;
          }
        }
      }
      // Stream closed cleanly without 'done' -> server recycled pod; reconnect from lastAppliedSeq
      throw new Error("Stream closed prematurely before terminal frame");
    } catch (err) {
      if (signal.aborted) return;
      attempt += 1;
      if (attempt > maxRetries) {
        opts.onPhaseChange("FAILED");
        throw err;
      }
      const capMs = Math.min(30_000, 500 * 2 ** (attempt - 1));
      const jitterMs = Math.floor(Math.random() * capMs);
      await new Promise((r) => setTimeout(r, jitterMs));
    } finally {
      if (watchdog) clearTimeout(watchdog);
      signal.removeEventListener("abort", onParentAbort);
    }
  }
}

Mandatory BFF / Server Streaming Headers

When proxying SSE through an Express/Fastify BFF, Nginx, or Envoy (Backend-for-Frontend Pattern, API Gateway Patterns), ensure responses always flush immediately and emit periodic heartbeats:


4. Production Failure Modes and UX Regressions

  1. Tying long-running generation lifetime to the open browser HTTP socket: If the worker aborts the LLM/tool pipeline the moment req.on("close") fires when a user switches networks in an elevator, the user loses 25 seconds of generated work. Decouple execution lifecycle (runs to completion or explicit user POST /runs/:id/cancel in a Background Job Processing worker) from view attachment (GET /runs/:id/events?afterSeq=N).
  2. Nginx / gzip compression swallowing tokens into 30-second bursts: Without X-Accel-Buffering: no and Cache-Control: no-transform, reverse proxies buffer the first 4–16 KB of SSE text before flushing anything to the browser, making "streaming UI" appear completely frozen until completion.
  3. Half-open TCP zombies with zero UI feedback: Mobile clients drop packets silently without sending a TCP FIN. Without a client-side heartbeatTimeoutMs watchdog (25 s), fetch() waits indefinitely while showing an eternal spinner. Pairing 12 s server : ping frames with a 25 s client watchdog detects frozen sockets deterministically.
  4. 60 FPS React re-render jank during high-speed token streaming: Calling setState on a massive Markdown document on every 2 ms token bursts the browser main thread. Buffer incoming text deltas in a mutable ref or store queue and commit UI updates on requestAnimationFrame (~16 ms batch intervals), parsing Markdown incrementally by paragraph block.

5. Agent Implementation & Output Contract

When an automated coding agent implements AI chat interfaces, agent workflow progress feeds, or real-time operational dashboards, enforce these invariants:

Required Architectural Components

  1. Separated command & stream attachment: Create an idempotent start mutation (POST /api/streams/..., returning { streamId } with Idempotency-Key support) distinct from the resumable stream subscription endpoint (GET /api/streams/:id/events?afterSeq=N).
  2. Monotonic sequence envelope + server replay buffer: Every emitted event carries a strictly increasing integer seq backed by a bounded Redis Stream or DB event log (TTL \ge 15 minutes) so reconnects recover every missed frame cleanly.
  3. Resilient client state & visible reconnection UX: Surface explicit connection phases (STREAMING, RECONNECTING, COMPLETED, FAILED) plus a user-triggered Cancel (AbortController + POST /cancel) and Retry from checkpoint button.
  4. Clean proxy headers & keepalive: Emit Cache-Control: no-cache, no-transform, X-Accel-Buffering: no, and : ping keepalive comments every 10–15 s on idle intervals.

Mandatory Verification Tests to Generate

  1. Mid-stream drop & replay test: Stream frames seq = 1..3, drop socket mid-stream while producer continues emitting 4..6, reconnect client with afterSeq = 3 -> assert UI renders 1..6 in exact order with zero duplicates and zero gaps.
  2. Heartbeat watchdog recovery test: Open SSE connection, withhold : ping and data frames beyond heartbeatTimeoutMs -> assert client aborts frozen socket, enters RECONNECTING, and resumes cleanly with jittered backoff.
  3. Proxy non-buffering & terminal cleanup test: Assert response headers include no-transform / X-Accel-Buffering: no, HTTP 401/403 terminates retry loop immediately (FAILED), and component unmount triggers AbortController.abort() without memory leaks.