Streaming

Streaming lets your AI agent publish its output to an Ably channel, so every subscribed client receives it as the model produces it. A client that disconnects can reconnect and receive what it missed, and a client that reloads replays the channel from the point your own store reached. Ably orders and delivers every message on a fault-tolerant global network, at low latency from any region. Message ordering covers what order means when publishing from different regions.

Why an HTTP response is not enough

A streamed HTTP response serves one client. A second device the user picks up sees nothing, because the response belongs to the request that opened it, and a dropped connection loses the rest of the response. An Ably channel delivers the response to every subscribed client, and a client that disconnects receives the rest of it when it comes back.

Understand the model

One conversation is one Ably channel. The agent and the client each create a transport for the channel, with a codec that turns your framework's events into Ably messages and back:

• The agent pipes the model's output to the transport. The codec appends each streamed part of the response to one message, so the response grows in one message on the channel.
• The client subscribes to receive every message on the channel as a delivery: the decoded event, and the Ably message that event was decoded from.
• The client wakes the agent with an HTTP request to an endpoint you own. The prompt can go in that request, or the client can publish it to the channel with send().
• The channel only holds the user's input messages if someone publishes them with send(). If the prompt goes in the HTTP request, the agent receives it first and must publish that prompt to the channel in order for multi-client fan-out to work.

Everything the agent and the client publish is an ordinary channel message, so ordering, storage and replay come from the channel. The send() and pipe() calls take headers, which go on every message they publish, so a client can tell which prompt each response answers.

Client and agent example

The client wakes the agent with an HTTP request, which carries the user's message, and subscribes to the channel:

// Client code. The browser fetches a token, so it never holds an API key.
import * as Ably from 'ably';
import { createTransport } from '@ably/ai-transport';
import { openai } from '@ably/ai-transport/openai';

const ably = new Ably.Realtime({ authUrl: '/auth' });
const transport = createTransport({ client: ably, channelName: 'conversations:42', codec: openai });

transport.subscribe(({ event, message }) => {
  if (event !== undefined) render(event, message.extras?.headers?.requestId);
});

const requestId = crypto.randomUUID();
await fetch('/api/chat', { method: 'POST', body: JSON.stringify({ channelName: 'conversations:42', items, requestId }) });

The agent is a function that takes the channel name and the user's input. It loads the conversation from your store and pipes the model's output to the channel:

// Agent code. This runs on your own infrastructure, so it holds the API key.
import * as Ably from 'ably';
import OpenAI from 'openai';
import { createTransport } from '@ably/ai-transport';
import { openai } from '@ably/ai-transport/openai';

const ably = new Ably.Realtime({ key: process.env.ABLY_API_KEY });

async function respond(channelName, items, requestId, signal) {
  const transport = createTransport({ client: ably, channelName, codec: openai });

  // Optionally: publish the user's input to the channel
  await transport.send({ type: 'input', items }, { headers: { requestId } });

  const stream = await new OpenAI().responses.create(
    { model: 'gpt-5.5', input: [...(await loadConversation(channelName)), ...items], stream: true },
    { signal },
  );

  const { serial } = await transport.pipe(stream, { signal, headers: { requestId } });

  await transport.close();
  return serial;
}

pipe() resolves with the serial of the last message it published. Store it with the response, so a client that loads the conversation later can replay the channel from it.

Know who implements what

Streaming splits the work of a conversation three ways:

What you still own

The conversation lives in whatever database your application already uses. You are responsible for giving clients the stored conversation, usually over a REST endpoint you own, and for storing the serial of each response with it. A client then replays the channel from that serial with fromSerial, so it receives what arrived after your store, in order. You merge the decoded events into whatever your UI renders.

Get the prompt to the agent

You are also responsible for waking the agent, typically with an HTTP request to an endpoint you own.

The agent can read the prompt in two ways:

• The client sends the prompt in the HTTP request that wakes the agent. This is good if the agent is the only one that needs to see the prompt, because the agent does not need to subscribe to the channel.
• The client publishes the prompt to the channel, and the HTTP request carries only the channel name and the requestId. The agent reads the prompt from the channel and finds it by its requestId header, either with a history replay that stops at the prompt with untilEvent, or by reading the channel's history(). This is good if other subscribers to the channel need to see the prompt in realtime.

How to get started

Start with one of the quickstarts, for the Vercel AI SDK or OpenAI.

A codec encodes your events into Ably messages and decodes them back. The SDK ships codecs for the Vercel AI SDK's UI message stream, for the OpenAI Responses API, and for AG-UI events. For any other format, write a custom codec.

Read next

• Get started with the Vercel AI SDK: a working chat app whose conversation stays in your own store.
• Get started with OpenAI: the same app with the Responses API.
• Write a custom codec: a codec for a provider AI Transport does not ship one for.
• History and replay: replay the channel from your own store.