AI Transport

AI Transport is a durable session layer for AI applications. Your agent publishes its response into a durable session on Ably.

AI Transport helps you build AI applications that are resilient, resumable, mutli-user and multi-device. Agents and clients connect to a durable session on Ably, which outlives the connection that started it. This allows for multi-device access, and automatic reconnection, resume, and recovery from network drops.

AI Transport is the delivery infrastructure. You keep your model provider, whether that is OpenAI, Anthropic, Google, or a model you host, and your prompts, orchestration, and hosting stay where they are. AI Transport ships a drop-in transport for the Vercel AI SDK and a codec for the OpenAI Responses API. The Core SDK composes with other frameworks, including Vercel WDK and Temporal.

What you gain over HTTP streaming

Most AI frameworks send one HTTP request per turn and stream the AI model's response back over the HTTP response. This ties the AI model's response to the HTTP request, other clients cannot read the response, and if the HTTP connection drops the response is lost.

Scenario	Direct HTTP streaming	AI Transport session
The connection drops mid-response	The response is lost, the model keeps generating tokens that are discarded, and the user sees an error and starts again.	The agent keeps publishing into the session. The client reconnects and resumes from the last token it received.
The user opens the conversation on a second device	The stream belongs to the device that opened it, so a second device cannot read it.	Every device attaches to the same session and sees the conversation as it streams.
The user reloads the page mid-answer	The reload discards the request and the partial answer with it.	The client reattaches and reads history and the in-flight response from the session, where the answer is a single message that grows as tokens arrive.
The user presses stop	Closing the connection is the only signal available, and the agent cannot tell it apart from a network drop.	The client publishes a cancel signal. The agent's current run ends and the session stays open.
A second message arrives before the first answer finishes	A second request opens a second stream, and the client interleaves two responses or drops one.	Each turn is its own run on the same session, with its own stream and cancel handle.
The agent process restarts mid-answer	The outbound stream ends with the process, and the client is left displaying a truncated response.	Where the agent runs inside a workflow engine, the retried step publishes into the same session.

A dropped connection loses a response whether the application has one user or a million.

Solving these individually means building a buffer so a stream can resume, a database for conversation state, and a queue or a second WebSocket for the client to signal on. It also means a reconciliation step that merges stored state with the live stream whenever a client joins. None of it is specific to the AI product, and HTTP streaming and AI goes through each limitation and the workaround code it requires.

What the session adds beyond delivery

A session runs on an Ably channel, with a conversation model on top of it. The rest of Ably's channel features work on that channel, and the SDK builds a conversation tree from its message log, which is what makes branching and editing possible.

Each of the following is part of the session:

What you get	How it works
Branching, edit, and regenerate	Editing a message or regenerating a response forks the conversation instead of overwriting it. The tree keeps every branch, and a view selects one path through it per client.
Agent and client presence	An agent reports whether it is thinking, streaming, idle, or offline, and clients that enter presence appear alongside it. An agent can also watch presence and stop work when nobody is connected to read the answer.
Shared live state	The agent reacts to what the user is doing, such as the record they have selected, without polling or extra tool calls. Client and agent read and write the same state over the session.
Interruption and steering	A client sends a follow-up into the run that is already streaming, or cancels it and re-prompts.
Human-in-the-loop approval	A run suspends until a tool call is approved. The request waits in the session, so any user can answer it from any device, minutes or hours later.
Chain of thought	Reasoning arrives as a separate stream within the same run, so a UI can render it beside the response.

Each of these needs a second transport alongside the stream if you build it on direct HTTP streaming.

Keep the stack you have

In the agent, pipe the stream you already build into a run on the session instead of returning it as the HTTP response body:

JavaScript

// Agent-side, in place of `return result.toUIMessageStreamResponse()`:
import * as Ably from 'ably';
import { streamText, convertToModelMessages } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import { Invocation } from '@ably/ai-transport';
import { createAgentSession } from '@ably/ai-transport/vercel';

const ably = new Ably.Realtime({ key: process.env.ABLY_API_KEY });

export async function POST(req) {
  const invocation = Invocation.fromJSON(await req.json());
  const session = createAgentSession({ client: ably, channelName: invocation.sessionName });
  await session.connect();
  const run = session.createRun(invocation, {}, { signal: req.signal });

  try {
    // Page the view back through history to rebuild the prompt. Do this before
    // run.start(), or it waits for the triggering input to arrive live.
    while (run.view.hasOlder()) await run.view.loadOlder();
    const messages = run.view.getMessages().map(({ message }) => message);

    await run.start();
    const result = streamText({
      model: anthropic('claude-sonnet-4-20250514'),
      messages: await convertToModelMessages(messages),
      abortSignal: run.abortSignal,
    });
    const streamResult = await run.pipe(result.toUIMessageStream());
    await run.end({ reason: streamResult.reason });
  } finally {
    await session.end();
  }

  return Response.json({ runId: run.runId, invocationId: run.invocationId });
}

The route returns the run's identifiers. The client reads the answer from the session. Where an answer takes longer than your function's timeout, durable execution moves the run into a workflow engine and the route returns before it finishes.

In the browser, a client attaches to that same session by name:

JavaScript

// Client-side.
import * as Ably from 'ably';
import { createClientSession } from '@ably/ai-transport/vercel';

const ably = new Ably.Realtime({ authUrl: '/auth' });
const session = createClientSession({ client: ably, channelName: 'conversations:42' });

await session.connect();

Prompts, tool definitions, model calls, and rendering are untouched. AI Transport implements Vercel AI SDK's ChatTransport interface, so useChat accepts it as its transport.

Two one-time setup steps apply. Browser clients connect with token authentication rather than an API key, and the namespace your conversations live on needs one channel rule enabled, because AI Transport streams tokens by appending to a message.

The SDK is JavaScript and TypeScript, with React hooks for the client. The roadmap covers other languages.

Limits and dependencies

These apply to every application built on AI Transport:

Limit or dependency	Detail
Retention limits what the session holds	The session holds the conversation for as long as channel retention covers it, a window you set per namespace. Beyond that window you keep your own database. Persist each completed run, and the union of those runs reconstructs the conversation. Database hydration joins the stored history to the live session with no gaps and no duplicates.
Hydrated history is linear	Messages you have persisted yourself cannot be edited or regenerated, so branching applies to the part of the conversation the session still holds.
Session content is visible to subscribers	Every message reaches every subscriber whose token capability allows it, tool inputs and outputs included. Scope capabilities per namespace so a token only grants access to the sessions that client should read.
A network hop and a third-party dependency	Conversations go through Ably instead of your own infrastructure, so every publish makes a round trip. Append rollup batches tokens, so that round trip happens once per batch rather than once per token.

Going to production covers limits, retention, monitoring, auth hardening, and pricing.

How a session is built

Every session is backed by an Ably channel: a durable, ordered, append-only log that any client or agent attaches to by name. Messages outlive the connection, device, or process that published them, and they have a total order. A client that drops reattaches and resumes without gaps or duplicates.

Any client or agent publishes, which puts cancel and steering on the same path as tokens. The SDK layers a conversation on top of that log:

A codec maps your framework's event types onto channel messages. Tokens stream by appending to a single message, so a client arriving late reads one assembled response.
The transport does not constrain what publishes into it, so a custom codec adds support for a framework with no bundled adapter.
Client and agent sessions own attachment, the run lifecycle, and cancel routing.
React hooks cover streaming, pagination, and branch navigation in the UI.

When you do not need a durable session

A single-turn chatbot does not need a durable session. If a user asks one question and never returns to the conversation, direct HTTP streaming is enough.

Platform guarantees

Ordering, persistence, replication, and regional failover are guarantees of the Ably platform, which is designed for 99.999% global service availability. They apply to AI Transport in the same way as to every other Ably product.

Ably is SOC 2 Type II certified and HIPAA compliant, and operates a bug bounty program.

Start building

Choose where to go next:

Get started

Build a working chat application with the Vercel AI SDK.

Features

Streaming, branching, presence, cancellation, and the rest.

Concepts

Sessions, runs, and the conversation tree behind the SDK.

Going to production

The checklist to work through before you ship.