History and replay

The Ably channel carries the live conversation messages to clients in realtime. The channel has a configurable retention period. To keep the conversation history for longer, hold it in your own database. The transport replays messages from the Ably channel, starting from the last point stored in your database.

// Client code.
import { fromSerial } from '@ably/ai-transport';

const subscription = transport.subscribe(handler, {
  history: { replay: fromSerial(stored.serial) },
});

How it works

When a subscription asks for history, the transport:

1. Automatically buffers live messages.
2. Reads channel history, and finds the message either by serial (fromSerial) or by a field on the message (untilEvent).
3. Sorts messages into channel order, and joins them with the live buffer.
4. Delivers all messages in channel order to the subscribe handler.

The SDK includes two common replay functions: fromSerial and untilEvent. You can write your own, because a replay function receives a page of history, paginates history until it has found the messages it needs, and returns everything that should be replayed.

Put the prompts on the channel

If neither the client nor the agent publishes the prompt, a client that rebuilds the conversation from the channel's history only gets the responses and not the prompts. So if your client is relying on the channel history (not your store) to load multiple prompts and responses then those prompts and responses must be on the channel.

Join your own store to the channel

Most applications keep the conversation in their own database, for history longer than the configured retention period.

When you write messages to your database, you can also write the Ably channel serial of that message. This gives you a channel serial that you can use as the position on the channel that your clients need to replay from, after they load the conversation history you have stored in your database.

The transport pipe() and send() methods both return the channel serial of the last message they published:

// Agent code.
const { serial } = await transport.pipe(stream, { signal: req.signal });
await saveResponse(channelName, response, serial);

When the client loads the conversation, it renders the conversation messages from your database and subscribes from the stored serial. fromSerial returns the messages after the serial: by default it uses the option inclusive: false, which means the message identified by the serial passed to fromSerial is not included in the replay:

// Client code.
import { fromSerial } from '@ably/ai-transport';

const stored = await loadConversation(channelName);
render(stored.messages);

const subscription = transport.subscribe(apply, {
  history: { replay: fromSerial(stored.serial) },
});

Replay from an event

A client that does not hold a serial can replay from an event it recognises with untilEvent. The predicate receives each decoded message, so it can match on the event's fields, on a header, or on the Ably message. The replay returns the newest matching delivery and everything after it:

// Client code, with the Vercel AI SDK codec.
import { untilEvent } from '@ably/ai-transport';

transport.subscribe(apply, {
  history: { replay: untilEvent(({ event }) => event?.type === 'start') },
});

Pass inclusive: false to replay only the deliveries after the match.

Check the result of a replay

The subscription carries a replayed promise. It resolves once the replay of history and buffered live messages has completed, which indicates that the replay is complete and the subscribe handler is now processing live channel messages as they arrive. The promise also reports the number of deliveries replayed, and whether the replay function found its point:

// Client code.
const { replayed, found } = await subscription.replayed;
if (!found) showNotice('Some earlier messages are no longer on the channel.');

found is false when history ran out, or the replay reached maxPages, before fromSerial reached its serial or untilEvent matched. By default the replay still delivers what it read. Pass onExhausted: 'error' to fail the replay instead, and maxPages to stop after a number of pages.

Write your own replay function

A replay function receives the newest page of history, reads older pages with page.next() as far as it needs, and returns the deliveries that should be replayed. The transport ensures the messages are delivered to the subscribe handler in the correct order, so the replay function does not need to do any sorting. This example pages history until there are no more messages, and replays every message found in the channel history:

// Client code.
transport.subscribe(apply, {
  history: {
    pageSize: 200,
    replay: async (firstPage) => {
      const deliveries = [];
      let page = firstPage;
      for (;;) {
        deliveries.push(...page.items);
        if (!page.hasNext) return deliveries;
        page = await page.next();
      }
    },
  },
});

To report back to the replayed promise if the target message was found, return { deliveries, found } instead of the array.

Read older messages

To show older messages when the user scrolls back, read history with transport.history(). It reads backwards from the point where the channel attached and returns the newest page, oldest first within the page. next() returns the next older page:

// Client code.
let page = await transport.history({ limit: 30 });
renderOlder(page.items);

async function loadOlder() {
  if (!page.hasNext) return;
  page = await page.next();
  renderOlder(page.items);
}

In React, useHistory keeps the pages it has read in one list, and loadMore adds the next older page to the front:

import { useHistory } from '@ably/ai-transport/react';

function Earlier() {
  const { items, hasNext, loading, loadMore } = useHistory({ limit: 30 });
  // ...
}

History is decoded through the same codec as live delivery. With a codec built with defineCodec, a message the handler has already received comes back with event set to undefined, so its text is not duplicated.

Load context on the agent

The agent loads the conversation it sends to the model from your own store, not from the request body. The client's POST carries only the new message. The agent can also read the channel with history() when it does not keep a store.

Edge cases and unhappy paths

• The replay buffer and the live buffer have no size limit. Set maxPages on fromSerial or untilEvent for a long conversation.
• The replay reads history from the point where the channel attached. Only the first subscription with history on a transport has no gap between history and live delivery. A later subscription with history, after the channel has attached, misses the messages that arrived between the attach and that subscription.
• One replay runs at a time on a transport. A second subscription with history waits for the first replay to finish.
• History reaches back as far as the channel's retention period. The appends rule that AI Transport requires persists messages, so the period depends on your package.
• If a page cannot be fetched after retries, or the replay function throws, replayed rejects with SessionHistoryFetchFailed. The handler still receives live messages, with a gap where the replay would have been.
• Unsubscribing or closing the transport during a replay stops it, and replayed rejects with OperationCancelled.
• A client without the history capability cannot replay or read history. Capability scoping is part of authentication.

FAQ

Do I need a database for conversation history?

Not for messages the channel still holds. The channel stores the conversation for your retention period. Add a database when you need conversations to last longer than that, or for search, analytics and reporting, and join it to the channel with fromSerial.

Why does the message at my stored serial not replay?

fromSerial assumes your store already holds the message at the serial, so it returns only the messages after it. Pass inclusive: true to replay that message too.

Can I replay history in more than one subscription?

Yes, but only the first subscription with history on a transport is gap free. The replay reads from the point where the channel attached, so a later subscription can miss the messages that arrived between that attach and the new subscription. To replay again, for example after a discontinuity, unsubscribe and subscribe again from the serial of the last message you received.

Does replay deliver a streaming response more than once?

No, with the codecs that ship with the SDK or a codec built with defineCodec. Those codecs track the text they have delivered for each streamed message, so a response that both history and live delivery carry reaches the handler once, with each piece of text once.

Related features

• Reconnection and recovery: replay the messages missed during a discontinuity.
• Token streaming: how a streamed response is stored as one message.
• Transport API reference: subscribe, fromSerial, untilEvent and history.