Streaming is a state machine, not a string

The first streaming implementation everyone writes is four lines:

for await (const event of stream) {
  if (event.type === "content_block_delta" && event.delta.type === "text_delta")
    process.stdout.write(event.delta.text);
}

It works. Text appears. Ship it.

Then it meets reality: users on trains, users who close the tab, responses that contain tool calls, responses that contain JSON, and a load balancer with a 30-second idle timeout. Every one of those breaks a different assumption in those four lines.

The mental correction that fixes most of it: a stream is not a string arriving slowly. It is a state machine emitting events, and some of those events are not text.

The events you're ignoring#

A Claude response stream is a sequence of typed events describing a document being built. Content blocks open, receive deltas, and close. The message itself opens and closes with metadata around it.

The four-line version handles exactly one delta type and drops the rest on the floor. If your response includes extended thinking, tool calls, or server-side tool use, those blocks are streaming past unhandled.

The block-oriented shape matters because blocks arrive interleaved and indexed. A response can open a text block, close it, open a tool_use block, and open another text block afterwards. Accumulating everything into one string buffer flattens structure you will need later.

type Block = { type: string; text: string };
const blocks = new Map<number, Block>();
 
for await (const event of stream) {
  switch (event.type) {
    case "content_block_start":
      blocks.set(event.index, { type: event.content_block.type, text: "" });
      break;
    case "content_block_delta": {
      const block = blocks.get(event.index);
      if (!block) break;
      if (event.delta.type === "text_delta") block.text += event.delta.text;
      if (event.delta.type === "input_json_delta") block.text += event.delta.partial_json;
      break;
    }
  }
}

The SDK will do this for you — stream.finalMessage() returns the assembled Message with all blocks in place, and you should use it rather than reimplementing assembly. Handle events for the live rendering; use finalMessage() for the authoritative result.

Partial JSON is not JSON#

When a tool call streams, its arguments arrive as input_json_delta fragments. Concatenating them mid-flight gives you strings like this:

{"query": "postgres full text sea

That is not JSON and will not parse. If you want to render tool arguments as they arrive — showing a search query forming in the UI, say — you need either a tolerant incremental parser or a rule that you only render fields once they're structurally closed.

The pragmatic answer for most applications: don't. Show "Searching…" while the tool call streams, wait for content_block_stop, then parse the complete input. The visual gain from streaming a JSON blob is close to zero and the failure modes are numerous.

If you genuinely need it — long structured outputs where users benefit from watching fields fill in — use a streaming JSON parser that emits values at completed key boundaries, and never JSON.parse a buffer you haven't seen close.

Reconnects: the stream has no replay#

This is the one that bites in production, and it applies to any SSE-based protocol.

If the connection drops and you reconnect, you get events from the reconnect point onward. Everything emitted during the gap is gone. There is no cursor, no resume token, no replay.

For a single messages.create stream, the recovery is to retry the whole request — annoying but simple, and prompt caching means the retry is cheap.

For long-lived agent sessions, dropping the gap is not acceptable, and the pattern is consolidation: open the new stream first, then fetch the event history through a separate paginated endpoint, and dedupe by event ID as the live stream catches up.

async function* consolidate(client, sessionId: string) {
  // 1. Open the stream FIRST — it starts buffering server-side.
  const stream = await client.beta.sessions.events.stream(sessionId);
 
  const seen = new Set<string>();
  // 2. Then read history, covering the gap.
  for await (const event of client.beta.sessions.events.list(sessionId)) {
    seen.add(event.id);
    yield event;
  }
  // 3. Tail the live stream, skipping what history already gave us.
  for await (const event of stream) {
    if (!seen.has(event.id)) {
      seen.add(event.id);
      yield event;
    }
  }
}

The ordering is the whole trick. Open the stream before fetching history — the other way around leaves a window where events fall between the end of your history page and the start of your subscription.

The same ordering rule applies on the happy path: open the stream before sending the message that triggers work. Send first and the agent may emit several events before your consumer attaches.

Backpressure, or: who is waiting for whom#

for await looks synchronous and hides an important property: if your loop body is slow, you are not consuming the stream. Do a database write per token and the socket buffer fills, TCP backpressure kicks in, and you are now the bottleneck on a response you're paying per-token for.

Decouple consumption from processing. Read events as fast as they arrive, push them onto a queue, and process from the queue.

The same applies to the browser. Sixty DOM updates per second from token-level re-renders will drop frames on anything mid-range. Buffer to an animation frame:

let pending = "";
let scheduled = false;
 
function push(text: string) {
  pending += text;
  if (scheduled) return;
  scheduled = true;
  requestAnimationFrame(() => {
    appendToDOM(pending);
    pending = "";
    scheduled = false;
  });
}

Users cannot perceive the difference between per-token and per-frame updates. Your frame budget can.

The abandoned request#

A user starts a long generation and closes the tab. What happens?

If nothing in your code notices, the model keeps generating, you keep paying, and the tokens go nowhere. On a chat product with any real traffic this is a measurable fraction of spend.

Wire the abort signal all the way through — from the HTTP request lifecycle, to the SDK call, to the generation:

export async function POST(req: Request) {
  const controller = new AbortController();
  req.signal.addEventListener("abort", () => controller.abort());
 
  const stream = client.messages.stream(
    { model: "claude-opus-5", max_tokens: 64000, messages },
    { signal: controller.signal },
  );
  // ...
}

Check that it actually works by opening a stream, killing the client, and watching whether your usage metrics stop. Most implementations that think they handle cancellation drop the signal at some framework boundary.

Timeouts that aren't timeouts#

Last one, and it's subtle: HTTP client timeouts in most libraries are per-read, not wall-clock. They reset every time a byte arrives.

A stream trickling one heartbeat every twenty seconds will never trip a 60-second read timeout, even if it runs for an hour. If you need a hard ceiling, track elapsed time yourself at the loop level and abort explicitly. Neither requests nor httpx nor most JS clients give you a total-duration timeout for free.

What good looks like#

Streaming done properly is roughly:

1. Handle every event type you might receive, not just text_delta.
2. Use the SDK's finalMessage() for the authoritative result; use events for live rendering only.
3. Never parse partial JSON — wait for the block to close.
4. Consolidate on reconnect: stream first, then history, dedupe by ID.
5. Buffer rendering to a frame; don't do slow work in the read loop.
6. Propagate cancellation end to end, and verify it with a real test.
7. Enforce wall-clock deadlines yourself.

None of that is exotic. It is the same discipline any long-lived connection demands — the novelty is only that the bytes cost money and the protocol has structure worth respecting.