AI Transport

The agent publishes to an Ably channel, and every client subscribed to that channel receives the output in realtime. A user can follow a response on a laptop and a phone at the same time, and several users can participate in a single conversation. A client that reloads or loses its connection can pick up where it left off. The conversation, the agent's status and the artifacts the agent creates all coexist on a single channel.

What AI Transport is

Your application interacts with the Ably channel directly through the Ably Pub/Sub SDK, so you can use the full range of Ably channel features.

The AI Transport SDK packages add helpers that work with the channel. The SDK provides an adaptor for each agent framework, which publishes the agent's output to the Ably channel. On the client, a decoder reads the messages from the channel and returns the agent's events for your UI to use. The adaptor and the decoder share a codec, which maps each event to the Ably messages that carry it.

AI Transport ships with built-in support for:

• Vercel AI SDK, and its useChat hook through the chat transport
• OpenAI Responses API
• AG-UI
• Custom event formats, which you build with the codec builder

Conversation, awareness and artifacts

AI applications typically have three types of data: the conversation, awareness, and artifacts.

The conversation consists of the prompts a user sends and the responses from the agent, including any tool approval requests, or other requests the agent makes of the user.

Awareness lets your agent keep users up to date as it works. The agent makes its status, plan, and progress available to the user, and updates them in realtime.

Artifacts are the reusable output that the agent creates in response to the user's prompt, such as documents, objects, web pages, plans, reports and images. AI Transport helps your application expose artifact state, and live artifact updates, to users.

One Ably channel

The conversation, awareness and artifact features can all make use of the same Ably channel. Messages carry the conversation. Presence, ephemeral messages, and LiveObjects carry awareness, and LiveObjects also carries artifacts. A client can make use of all of these features over a single Ably connection and a single channel.

An application with several agents can have each agent publish to the same channel, and the client subscribes once to receive every agent's output.

Use token capabilities to control what a client has access to.

Connect the agent and the client

The agent, Ably, and the client connect in three directions:

• The agent can use the Ably realtime client to publish streamed output to Ably, or the HTTP client to make one-off reads and writes to Ably features.
• Ably delivers everything the agent publishes to each client subscribed to the channel.
• The client typically sends the prompt to the agent using the existing HTTP endpoints that you already use. The client can publish prompts directly to the channel, but this is a less common pattern.

Typically, the agent does not receive messages unless it attaches to the channel, for example to subscribe or to enter presence. An agent can make use of the HTTP client to update the agent's status, or write an artifact to LiveObjects without attaching to the channel.

These sketches leave out authentication, your database and sending the prompt to the agent.

This agent code runs on your server, and pipes the model's stream to the conversation's channel through an adaptor:

// responseStream comes from responses.create with stream: true.
import { createRealtimeClient } from '@ably/pubsub-server';
import { openaiAdaptor } from '@ably/ai-transport/openai';

const agentClient = createRealtimeClient({ key: process.env.ABLY_API_KEY });
const adaptor = openaiAdaptor(agentClient.channels.get('conversations:42'));
await adaptor.pipe(responseStream, { signal: req.signal });

This client code runs in the browser, and subscribes a decoder to the same channel to pass each of the agent's events to your UI:

// The browser fetches a token from your auth route, so it never holds an API key.
import { createClient } from '@ably/pubsub-device';
import { openaiDecoder } from '@ably/ai-transport/openai';

const client = createClient({ authUrl: '/auth' });
const decoder = openaiDecoder();
decoder.onEvent(({ event }) => render(event));
await client.channels.get('conversations:42').subscribe(decoder.receive);

Works with your existing stack

Your database stays the source of truth for the conversation, and your database or blob storage keeps the durable copy of the agent's artifacts. The channel delivers the live output and keeps every client up to date. In the agent, you can pipe the stream of LLM events directly to the channel, and your model, prompts, tool definitions, hosting and UI code stay as they are. If your agent uses its own event structure, you can make use of a custom codec.

A Vercel AI SDK app can keep useChat on the client, and pass useChat the chat transport in place of its default HTTP transport.

Your agent can run on a workflow engine such as Temporal or Vercel WDK. With durable execution, each attempt that the engine runs publishes its output to the same channel.

Limits and dependencies

Before you ship, work through the production checklist for limits, retention periods, monitoring, auth hardening and pricing.

Platform guarantees

The Ably platform guarantees ordering, persistence, replication, and regional failover. Ably designs the platform for 99.999% global service availability, and delivers messages at low latency across regions. The platform's scalability is elastic. These guarantees apply to AI Transport in the same way as to every other Ably product.

Ably is SOC 2 Type II certified and HIPAA compliant, and operates a bug bounty program.

Pick where to start

Build a working app with your framework, or set up your Ably app first: