Build a chat agent on standard HTTP streaming, SSE, or a raw WebSocket, and the connection will drop mid-response, taking the partial output with it. There's no server-side session state to reconnect to, so the client re-prompts from scratch, asking the same question, getting the same clarifying questions. The model didn't forget; nothing outside that one connection held the conversation state.
Connection drops, page refreshes, and device switches all fail the same way: session state lives in the connection, not independently of it.
This demo shows how Ably AI Transport fixes it. Mike Christensen (Pub/Sub team lead at Ably) walks through a live multi-agent holiday planning app. It's built on a durable session, one that outlasts any single connection. That session solves the primitives most production teams end up building from scratch: barge-in, human handover, and multi-agent coordination. Those are covered later in this post, in the same order as the video's chapters.
Key takeaways
- Connection drops restart most AI streams from scratch. Ably AI Transport buffers session output in the channel, so clients reconnect and catch up without re-running inference.
- Barge-in requires a bi-directional channel. Server-Sent Events (SSE) can't distinguish a user interrupt from a network drop; AI Transport delivers cancel and redirect as explicit channel signals the agent acts on.
- Organization-side human handover, where a supervisor joins a live session on a different device hours later, is the HITL case most frameworks leave unsolved. AI Transport's durable session persists the pending approval in session history until the right person responds.
Why AI agent streams break in production
Connection drops mid-stream. Standard HTTP streaming stores no session state server-side. When the connection closes, the tokens generated during the gap disappear: the delivery layer was never asked to hold them. The client reconnects to an empty state and re-prompts.
Page refresh loses the stream. Most AI implementations store token state in the browser: React component state, a JavaScript variable tracking the partial response. When the page reloads, that state is gone. The agent has no awareness that the client disappeared mid-generation, and no mechanism to re-stream output that it already produced.
Device switches lose the session. Sessions are tied to connections, and connections are tied to devices. Move from laptop to phone, and the conversation doesn't follow. The new device has no path to the session's history.
All three share the same root cause. Generation state is coupled to a single delivery connection. Decoupling them, so session state outlasts any individual connection, fixes all three at once. For the user, that means no more lost context. For you, it means no more hand-built reconnection logic.
SSE, WebSockets, and durable sessions compared
SSE is a one-way HTTP stream: the server pushes tokens, the client only reads, and there's no return path for cancel or steer signals. WebSockets fix the one-way limitation with a persistent, bi-directional socket, but that socket is still the session. If it drops, the server has nothing to replay what it already sent. A second device opening its own connection can't read the first device's conversation, either.
A durable session, the model that AI Transport implements, separates the conversation from any single connection entirely. The agent publishes into the session itself, rather than a response body or a socket. Any device, WebSocket connection, or reconnect attempt reads from that same session, so connection drops and device switches stop being failure modes.
That durability isn't free: publishing into a durable session adds a network hop that a direct SSE response doesn't have. For a single-turn, single-device chat, that's overhead with no payoff. But for sessions that have to survive a reconnect, follow a user across devices, or stay open for a human to join hours later, it's essential.
| Property | SSE | WebSockets | Durable session (AI Transport) |
|---|---|---|---|
| Return path from client to server | None (server to client only) | Yes, native | Yes (via session publish) |
| Survives a connection drop | No (the stream is lost) | No (the socket closes and state is lost) | Yes (retained independently of the connection) |
| Multi-device or multi-tab | No | No (each socket is its own session) | Yes (any device attaches to the same session) |
| Works through corporate proxies and load balancers | Often blocked or timed out | Frequently blocked entirely | Falls back through WebSocket, HTTP streaming, then long-polling |
| Barge-in implementation effort | Needs a separate side channel | Native, but you build session state yourself | Native, with session state included |
When a durable session is worth it, and when it isn't
A single-turn chatbot with no follow-up and no multi-device requirement doesn't need one. Plain SSE is enough, because there's nothing to reconnect to.
A bi-directional feature, such as barge-in or live steering, can run on a raw WebSocket connection in some cases. This works only if it runs in a single browser tab in a single sitting, with no crash-recovery requirement. Losing state on a drop needs to be acceptable.
A durable session earns its place once any of the following apply:
- Users reconnect after a drop and expect to see what they missed.
- The same conversation needs to be visible on more than one device.
- An agent's work needs to survive a process crash or redeploy.
- A human needs to join a session hours after it started.
How Ably AI Transport handles connection recovery and session continuity
Server-side buffering and offset-based replay. Every token the agent publishes goes to the session channel as it's generated, regardless of whether the client is connected. On reconnect, AI Transport uses untilAttach to deliver everything published during the gap, in order, before the live stream resumes. The LLM never re-runs; the client catches up.
Session on the channel, not the connection. The session lives in the channel, not in the connection that opened it. Any device subscribing to the same channel name joins the same session: full conversation history, followed by the live stream from its current position. Two browser tabs, a laptop and a phone, a page reload mid-response: all receive the same unbroken state.
Channel history for context. When a client has been offline beyond the live recovery window, channel history provides the full conversation. Clients load older messages using view.loadOlder(), paginating back through the session until they have the full context. For users who are offline entirely, push notifications via FCM, APNs, or Web Push can deliver agent completions when they return. Push notification delivery is currently Partial in the feature set.
In the demo, Mike refreshes the page mid-stream, and the response picks up exactly where it stopped. Two windows open side by side show the same in-progress response, updating simultaneously.
Session continuity is the infrastructure layer. What happens on top of it: how users interact with agents in motion, how human operators step in, how multiple agents coordinate, depends on it being in place.
The next four sections cover the interaction patterns the demo demonstrates: what the user sees while the agent works, how they interrupt or redirect it, how a human operator takes over with full context, and how multiple specialised agents surface progress independently. All four require the session to be live and visible.
Agent progress visibility: what the user sees while the agent works
A user can only meaningfully interrupt an agent they can see working. Progress visibility is the prerequisite for both barge-in and human handover. Without visibility, users have no basis for interrupting: they're canceling a process they can't see, with no information about whether to wait or redirect.
The demo surfaces four types of progress signal. Token streaming shows what the orchestrator is generating. Ably LiveObjects carries the structured progress state from each of the three specialist agents: flights, hotels, and activities. Presence shows which agents are active in the session, and task history shows what each has completed.
Each signal comes from a different source, and each arrives independently. All three specialist agents publish their progress directly, without routing through the orchestrator. So the user sees the live state from each agent simultaneously. Each agent also converts its raw query parameters into natural language using a separate model call. Progress cards show "Searching for direct flights on the 14th" rather than a query object. That's what makes barge-in useful. The user's decision to interrupt is based on accurate realtime information, not a stale snapshot.
Barge-in: how users interrupt and redirect agents mid-response
In a talk about Ably's customer discovery research, Ably CEO Matthew O'Riordan explains why asynchronous agent experiences need explicit interruption controls. One research participant had disabled user input entirely because SSE can make a user's stop signal look the same as a network drop. So there's no safe way to act on it.
AI Transport changes this because the session is bi-directional. User input arrives as a specific signal on the session, not a connection side effect. So the agent can act on it reliably while remaining live.
The video shows two patterns that were available at the time of the demo: cancel-then-send, and send-alongside.
Cancel-then-send is the more common of the two. The client filters session.view.runs() for the active run and calls session.cancel(runId). That fires the agent's abortSignal, stops the LLM stream, and ends the run with reason 'cancelled', while the session itself stays intact. In the demo, Mike says "I want to visit a museum" while the activities agent is mid-search. That's a redirect: the original task has no remaining value, so the client cancels that run. The new message then starts a fresh run on the museum query.
Send-alongside is the alternative. It starts a new run without canceling the active one, so the application can display both outputs side by side. Both runs multiplex on the same session, each with its own stream and its own cancel handle. Call session.cancel(runId) against the specific run if one later becomes unnecessary. Use this pattern for a follow-up or comparison when both responses remain useful.
AI Transport now also supports steering, which lets a follow-up fold into the still-running response instead of canceling it or running alongside it. See Interruption and steering for how all three compare. The choice between cancel-then-send and send-alongside depends on what you want the user to see.
For the full API reference, see the Interruption and barge-in docs.
Human-in-the-loop: getting full session context to an operator on any device
Most frameworks implement one variant of human-in-the-loop (HITL) and leave the other unsolved. But the distinction between them matters in production.
User-side HITL is the pattern where the agent pauses and asks the user to approve an action before executing. For example, "Should I book this flight?". The user approves or rejects, and the agent continues. Almost every agent framework has this.
Organization-side HITL is the harder case. The agent needs to escalate to an internal supervisor: someone who may be on a different device, in a different time zone, and who might not respond for hours. This is the customer support scenario: a human agent takes over mid-conversation, with full context, without the user re-explaining anything. Most frameworks leave this unsolved.
AI Transport handles both through the same mechanism. The agent defines a tool with a gate that gets checked per call rather than executing automatically. When the LLM invokes that tool, and the gate says it still needs approval, the agent suspends the run instead of ending it. The pending tool call is then published to the session as a durable message.
Any connected client renders the pending approval and publishes the approval or rejection back to the session. A supervisor joining on a different device hours later reads the same pending request from channel history and resolves it the same way.
The approval is a durable channel message, not a live server process waiting to time out. The response triggers a continuation invocation under the same run ID, and the agent picks up where it paused.
Organization-side escalation is documented today: any client with publish capability can submit the approval. The docs cover the full flow from tool definition to continuation.
For the full implementation detail, see the Human-in-the-loop docs.
Multi-agent coordination and shared state via Ably LiveObjects
Routing all agent activity through a central orchestrator creates a bottleneck. Every progress update has to pass through the coordinator before it appears to the user. At the scale of a multi-step, multi-agent workflow, that lag accumulates.
This demo takes a different approach. The orchestrator delegates to three specialist agents: flights, hotels, and activities, all running concurrently. Each specialist publishes its progress directly to Ably LiveObjects - bypassing the orchestrator entirely for user-facing updates.
The orchestrator waits for final results. The user sees live progress bars from all three agents updating in realtime, independently.
LiveObjects carries more than progress signals. User selections (flight, hotel, and activities choices) are written to LiveObjects state the moment the user makes a choice. When the user later asks "What's my current itinerary?", the orchestrator reads directly from LiveObjects rather than reconstructing context from chat history. If the user deleted a selection outside the chat thread, the agent sees that immediately. The conversation is one interface to the system; the source of truth is the state.
This matters because the user-facing update rate is decoupled from the orchestrator's coordination cycle. Each agent surfaces progress as fast as it produces it, with no relay step in between.
And presence adds a further signal. Agents can check whether the user is actually connected before streaming. An agent completing a search while the user is offline can push a notification rather than stream into a disconnected channel.
You can learn more about Ably LiveObjects here.
Conclusion
Session continuity, barge-in, and human handover aren't features that sit on top of an AI stack. They're properties of the delivery layer underneath it. The session channel is what makes them composable: the same mechanism that replays tokens on reconnect makes a pending approval durable, and lets a supervisor join a live conversation hours after it started. Most teams reach for these patterns eventually. The question is whether you build them yourself or start with infrastructure that already has them.
Docs go deeper: Ably AI Transport documentation.
Frequently asked questions
How do I implement barge-in so a user can interrupt an AI agent?
Filter the session's active runs and call session.cancel(runId) on the one you want to stop. AI Transport publishes an explicit cancel signal on the session while the agent is still producing output. The agent's abort signal fires in response. Because it's a named signal on the session rather than a dropped connection, a network blip is never mistaken for a user cancellation.
// Cancel every run this client currently has active, then send the new message.
async function bargeIn(session, view, text) {
const myClientId = ably.auth.clientId;
const active = session.view.runs()
.filter((run) => run.status === 'active' && run.clientId === myClientId);
await Promise.all(active.map((run) => session.cancel(run.runId)));
return view.send(createUIMessageCodec().createUserMessage({
id: crypto.randomUUID(),
role: 'user',
parts: [{ type: 'text', text }],
}));
}
When should I use cancel-then-send versus send-alongside?
Cancel-then-send ends the current run before the application submits replacement input. Send-alongside keeps both runs active. Use cancel-then-send for redirects, such as Mike's museum request, where the original task no longer matters. Use send-alongside when both outputs remain useful, such as a follow-up question or a side-by-side comparison.
What's the difference between user-side and organization-side HITL?
User-side HITL asks the person already in the conversation to approve or reject an action. Organization-side HITL routes the decision to an internal supervisor who may not be connected when the pause happens. AI Transport stores an organization-side pending tool call in the session history until an authorized person resolves it. That lets an approver join hours later, without the original connection staying open.
How does a supervisor access the pending HITL approval if they're not already connected to the session?
The pending tool call sits in the session history, not in a server process waiting on a timeout. When the supervisor subscribes to the session, on any device, at any point after the run suspends, they receive the full history. That includes the pending approval. Push notifications through FCM or APNs can alert them that one is waiting. Once subscribed, they submit the approval, and a continuation invocation resumes the same run from where it paused.
How do multiple specialized agents publish progress updates to the same user session?
Each agent publishes directly to the session channel or Ably LiveObjects, without routing through the orchestrator. The user sees live updates from all agents simultaneously; the orchestrator only handles final results. This decouples the user-facing update rate from the orchestrator's coordination cycle.
Does Ably AI Transport work with any LLM or agent framework?
Yes. AI Transport operates at the session and delivery layer, below orchestration. That means it has no dependency on a specific LLM provider or agent framework. It ships a drop-in transport for the Vercel AI SDK and a codec for the OpenAI Responses API. For any other framework, the core SDK composes underneath it. The transport doesn't constrain what publishes into it, so a custom codec adds support for a framework with no bundled adapter.
How do I prevent rogue data mutations after stream cancellation?
Track each run's ID and its terminal reason, and discard any token that arrives tagged with a run that already ended cancelled. AI Transport reports the cancelled reason on the run's own lifecycle event rather than inferring it from silence. That makes the check exact rather than a guess.
const state = { text: '' };
const cancelledRunIds = new Set();
session.tree.on('run', (event) => {
if (event.type === 'end' && event.reason === 'cancelled') {
cancelledRunIds.add(event.runId);
}
});
function onToken({ runId, token }) {
if (cancelledRunIds.has(runId)) return;
state.text += token;
}
// A dropped connection alone never adds a run here; only an explicit
// cancel event does, so late tokens from a reconnect still render.
How do I cancel upstream LLM generation across a multi-agent session?
Call session.cancel(runId) from any client with publish capability. The agent's abortSignal fires, and the run ends with reason 'cancelled' on the shared session, the same mechanism used for a single-client run. What's different in a multi-agent or multi-device session is that every participant needs to see the run end. It's not just the client that issued the cancel. Subscribe to session.tree.on('run', ...) and check event.runId and event.type === 'end' to update each observer's own view. That view might be a supervisor dashboard, a second device, or a sibling agent watching the same conversation.
// Every participant watching the session - not just the client that
// issued the cancel - reacts to the run's shared lifecycle event.
session.tree.on('run', (event) => {
if (event.runId !== watchedRunId || event.type !== 'end') return;
if (event.reason === 'cancelled') {
markAgentPanelStopped(event.runId);
}
});
// Any participant with publish capability can trigger the cancel;
// every observer of the session sees the same lifecycle event.
await session.cancel(runId);
See Stop vs disconnect: canceling AI streaming (https://ably.com/blog/stop-vs-disconnect-canceling-ai-streaming) for the single-turn mechanics of run.end({ reason }). That post also explains why an explicit cancel signal, not a dropped connection, triggers it.
