Going to production

Before you ship AI Transport in production, walk through this checklist. It covers the operational concerns that come up across every feature.

Work through each item before traffic moves to production.

Pricing

AI Transport runs on Ably's usage-based billing. Ably meters published and delivered messages, channel time, and connection time. The exact rates depend on your package; contact Ably for enterprise pricing.

Two AI Transport patterns affect cost predictably:

Token streaming uses Ably's append rollup to compact many token events into fewer published messages. The default rollup window is 40 ms (about 25 messages per second per stream), and you can tune it.
Channel history retention determines how long a client can replay the conversation from Ably alone. A longer retention period means Ably stores more; pair it with an external store for long-lived conversations.

AI Transport pricing explains the costs of usage of the AI Transport product, stating how each operation contributes to your billable message total and discusses the levers that can be used to manage the cost. For a worked example, see the AI chatbot pricing example.

Limits and quotas

The platform limits page covers the hard limits in detail. The ones that matter most for AI Transport:

Connection inbound message rate. A single stream that exceeds the per-connection rate triggers rate-limiting. Configure the append rollup window to control the publish rate.
Channel message rate. Concurrent turns and multi-agent setups share the channel's rate budget.
Message size. Large tool outputs or attachments approach the per-message size limit. Stream large results or persist them externally and reference the URL.
Retention period. Configure your retention period through the channel rule settings. Plan for the longest gap you expect a client to replay from Ably alone.

Monitoring and observability

Watch three signals:

Channel publish rate per conversation, against the connection's inbound rate limit.
Pipe outcomes: resolved, rejected with OperationCancelled, or rejected with PipeFailed. A rising failure rate is your first signal that something is wrong with model orchestration, auth, or capacity.
Server-side LLM latency and error rate, from your model provider. The Ably side is decoupled from the LLM; track them separately.

Send these signals to your existing dashboards through Ably's integrations. Webhooks, Kafka, and other sinks expose channel activity. Ably also offers Stats API for usage data.

Auth hardening

The defaults are safe for development. For production:

Scope channel capabilities to the specific conversation channel (or a wildcard covering them) the user needs access to. publish, subscribe, and history only on channels the user owns.
Set clientId in the token rather than on the client. The Ably service verifies tokens; client-set IDs are unverified and spoofable.
Use short token lifetimes and authUrl (or authCallback) so the SDK refreshes tokens automatically. One-hour tokens are a reasonable default.
Implement token revocation for forced sign-out.

Set up authentication covers the token endpoint, capability scoping, and the token lifecycle. The multi-device and fan-out documentation covers client identity edge cases.

Data shipping and retention periods

Your own database stores the long-term conversation history, so your retention period only has to cover the longest gap you expect a client to replay from Ably alone.

How long the channel keeps messages depends on the channel's retention period. The appends rule persists messages for that period, and your own store covers anything older.
Store the serial that pipe() resolves with in the same write as the response, so a client can replay the channel from exactly where your store ends.
For analytics or search, stream channel activity to your data warehouse through webhooks or Kafka. This is one-way; AI Transport reads from the channel rather than the warehouse.

Compliance

Ably is SOC 2 Type II certified and HIPAA compliant.

For the application-side:

Message content, including tool inputs and outputs, is visible to every subscriber whose capability allows it.
For region-pinned data residency, scope capabilities to channels hosted in the right region of Ably's edge network.

Deployment notes

A few patterns that come up:

Serverless agents are fine. Each request creates a transport, pipes the response, and closes the transport. The conversation lives on the channel and in your store rather than in the agent process.
The agent owns its Ably.Realtime client. close() detaches the transport's channel and leaves the client connected, so create the client once per process and close it when the process shuts down.
after() (Next.js) or equivalent post-response continuation lets the HTTP route return before the model finishes streaming. Without it, the time available for streaming is limited by the request timeout.
Multiple agent regions are supported. Ably's infrastructure keeps state present in every region; an agent in one region publishes and clients in any region see the same state with low latency from their nearest edge.

Durable execution

If an agent's work spans multiple stages, such as inference, tools and follow-up inference, or exceeds a serverless function's runtime budget, run the agent inside a workflow engine such as Temporal or Vercel WDK. Each stage becomes its own retryable activity, and publishes to the same channel.

History replay

Clients replay the channel from your store with fromSerial. Plan for these edge cases and limits:

The replay buffer and the live buffer have no size limit. Set maxPages on fromSerial or untilEvent for long conversations.
History reaches back as far as your retention period, so a replay after a longer gap resolves with found: false.
Only the first subscription with history on a transport has no gap between history and live delivery. Subscribe with history once per transport, and subscribe again after a discontinuity.
Your agent has to be the only writer to the store clients replay from. A second writer puts messages between your stored history and the channel that the stored serial does not account for.