Conversation

The conversation consists of the prompts a user sends and the responses from the agent, including any tool approval or other request the agent makes of the user. Each new prompt or response goes on the end of the conversation. A device that was away, or that reloads, reads what it missed from your database and the channel.

The conversation is a log

AI Transport puts the conversation on an Ably channel, one channel per conversation. Prompts and responses are messages on the channel. A response is one message that grows as the agent appends each part of the model's output to it.

The same channel also carries awareness and artifacts.

Your database holds the durable copy of the conversation. The channel delivers each new message in realtime, and keeps its messages for the retention period of your channel rule. Each message on the channel has a serial that represents the message's position on the channel. Store that serial in your database so that a client that loads the conversation from your database can rewind to that serial and join live messages with the stored conversation.

The conversation gets these properties from the channel:

Connect the agent and the client

The agent, Ably, and the client connect in three directions:

• The agent publishes to Ably with a realtime client, through the adaptor. It uses an HTTP client for one-off reads and writes.
• Ably delivers everything the agent publishes to each client subscribed to the channel.
• The client typically sends the prompt to the agent over the HTTP endpoints that you already use. The client can also publish the prompt to the channel itself.

The agent's realtime client comes from @ably/pubsub-server. The agent needs a realtime connection, because the appends of a response stream over that connection in order.

This agent code creates the realtime client and gets the channel:

// This runs on your server, so it holds the API key.
import { createRealtimeClient } from '@ably/pubsub-server';

const agentClient = createRealtimeClient({ key: process.env.ABLY_API_KEY });
const channel = agentClient.channels.get(channelName);

The adaptor publishes messages to the channel without attaching, this means the agent does not need to subscribe to the channel.

Ably separates publishing from subscribing. Publishing alone does not attach the channel. Subscribing, entering presence and reading LiveObjects state in realtime do attach it, and an attached channel receives every message on the channel unless you limit it with channel modes.

Get the channel with the modes the agent needs before you pass it to the adaptor, because the modes apply when the channel first attaches. An agent that only streams responses can use the channel with no options. The modes you pass replace the default set, so you must list every mode the agent uses. For example, an agent that also enters presence, and does not need to receive messages, gets the channel with PUBLISH and PRESENCE:

const channel = agentClient.channels.get(channelName, { modes: ['PUBLISH', 'PRESENCE'] });

The agent's realtime client keeps its connection open until you call agentClient.close(), so you must close the client when the agent no longer needs it.

On the user's device, the client creates an Ably client with createClient({ authUrl: '/auth' }) from @ably/pubsub-device, so the browser fetches a token from your token route. The client subscribes a decoder to the same channel, which attaches the channel.

Stream a response to every client

The client subscribes to the channel and POSTs the prompt to your HTTP endpoint. The agent publishes the prompt, pipes the model's output to the channel, and returns the message serials for you to store.

This client code subscribes and sends a prompt:

// The browser fetches a token, so it never holds an API key.
import { createClient } from '@ably/pubsub-device';
import { openaiDecoder } from '@ably/ai-transport/openai';

const client = createClient({ authUrl: '/auth' });
const channel = client.channels.get('conversations:42');
const decoder = openaiDecoder();
decoder.onEvent(({ event, message }) => {
  if (event !== undefined) render(event, message.extras?.headers?.requestId);
});
await channel.subscribe(decoder.receive);

const requestId = crypto.randomUUID();
const items = [{ role: 'user', content: text }];
await fetch('/api/chat', { method: 'POST', body: JSON.stringify({ channelName: 'conversations:42', requestId, items }) });

This agent code publishes the prompt and streams the AI response. The agent sets the requestId header on the prompt and on every message of the response, so a client can match each response to its prompt:

// This runs on your server, so it holds the API key.
import OpenAI from 'openai';
import { createRealtimeClient } from '@ably/pubsub-server';
import { openaiAdaptor } from '@ably/ai-transport/openai';

export async function respond({ channelName, requestId, items }, signal) {
  const headers = { requestId };
  const agentClient = createRealtimeClient({ key: process.env.ABLY_API_KEY });
  const adaptor = openaiAdaptor(agentClient.channels.get(channelName));
  try {
    const { serial: promptSerial } = await adaptor.send({ type: 'input', items }, { headers });
    const stream = await new OpenAI().responses.create(
      { model: 'gpt-5.5', input: [...(await loadConversation(channelName)), ...items], stream: true },
      { signal },
    );
    const { serial: responseSerial } = await adaptor.pipe(stream, { signal, headers });
    return { promptSerial, responseSerial };
  } finally {
    agentClient.close();
  }
}

send resolves with the serial of the prompt's message. pipe resolves with the serial of the last message it published. You need to store each serial with its message, so a client that loads the conversation later can rewind the channel to the point your database reached.

Get the prompt to the agent

Typically, the client POSTs the prompt to your agent's HTTP endpoint, and the agent publishes the prompt to the channel with adaptor.send before it pipes the response. Every subscribed client then receives the prompt in realtime. You should store the prompt's message serial so that you have a point to rewind to even before the AI response has started streaming. A client that loads while the response streams can rewind the channel to just after the prompt, and receive the whole response.

By publishing the user's prompt from the agent, the prompt's Ably message carries the agent's connection identity and not the identity of the user. A client can match its own prompts by the requestId header. You choose how to show who wrote a prompt.

The client can also publish the prompt itself. The client wraps its channel in an adaptor, publishes the prompt with vercelAdaptor(channel).send(...), and then POSTs the message with the serial that send returned. In this path the client's token needs the publish capability. The client receives its own prompt back from the channel, unless you set echoMessages: false on createClient.

If the prompt goes only in the POST, and the agent doesn't publish the prompt to the channel, the other subscribed clients do not see the prompt, and your database does not have a rewind point until the response starts streaming to the channel.

What the SDK, Ably and your app do

The SDK, the Ably channel, and your application each implement a part of the conversation:

The decoder forgets a stream once the stream's last message arrives. Channel history that you read after that point gives you the whole response again, so you need to drop the repeated response by its serial.

Put your own messages on the channel

Your application interacts with the Ably channel directly through the Ably Pub/Sub SDK, so it can publish its own messages alongside the conversation. For example, the agent can set a title for the conversation after the first response. The AI Transport SDK publishes messages with the message name ai, so your messages should use a different message name.

This agent code publishes a title on the channel it already holds:

await channel.publish('title', { text: 'Weekend in Lisbon' });

The decoder on the client automatically handles messages that do not have the name ai. The message is exposed in the decoder's onEvent as { event: undefined, message }. This client code reads the title from message.name and message.data:

decoder.onEvent(({ event, message }) => {
  if (event === undefined && message.name === 'title') setTitle(message.data.text);
});

Read next

• Get started with the Vercel AI SDK: build a working chat app with a client and an agent.
• Token streaming: stream a response as appends to one message, and stop it.
• History and replay: store the serial, rewind the channel to it, and read older messages.
• Multi-device and fan-out: tell a device's own prompts from other devices' prompts, and join from another device.
• Reconnection and recovery: detect when the channel loses continuity, and recover from it.
• Agent adaptor: the API reference for send, pipe, and pipeEach.