Get started with OpenAI

What you build

A Next.js chat app where:

• The Responses API's output streams to every connected client in realtime.
• A second tab shows the same conversation, because every subscribed client receives the same messages.
• Reloading the page reads the conversation from your database, then replays what arrived on the channel after it, so a response still streaming carries on where it is.

The conversation lives in your own database. AI Transport carries the live traffic over an Ably channel, and you build your own interface. The one below is an example.

Prerequisites

• Node.js 22 or later.
• An Ably account with an API key.
• An OpenAI API key.

Install dependencies

Install the AI Transport SDK, the Ably Pub/Sub server and device SDKs, the OpenAI SDK, and Next.js, then add the scripts that run the app:

npm install @ably/ai-transport @ably/pubsub-server @ably/pubsub-device openai@^6 next react react-dom
npm pkg set scripts.dev="next dev" scripts.build="next build"

AI Transport does not depend on Next.js. This guide uses it as an example.

Set up authentication

Create an auth endpoint at /api/auth/token that returns an Ably token to the browser, with the client id and the channel capabilities the user needs, as described in Set up authentication. The client needs the subscribe and history capabilities on the conversation channels.

The client below fetches its token with authUrl: '/api/auth/token'. The agent's HTTP endpoint runs on your own infrastructure, so it uses the API key directly.

Configure the channel rule

AI Transport streams a response's text by appending each delta to a channel message. That needs the Message annotations, updates, deletes, and appends rule (mutableMessages: true in the Control API) on the channels your conversations use. Enable it once per Ably app through the dashboard, Control API, or CLI. The rule also persists the channel's messages, so a client that reloads can replay them.

Create the agent's HTTP endpoint

Create app/api/chat/route.js. The POST handler is the agent. It loads the stored conversation, publishes the new input to the channel with send, calls the Responses API with the stored items and the new input, and pipes the stream to the channel. The agent stores the input items with the serial that send returns, so a client that loads while the response streams replays the whole response. A small generator passes each event to the pipe and keeps the finished output items from the response.completed event. When the pipe resolves, the route stores the output items, with the serial of the last message the pipe published.

If the client closes the request, req.signal cancels both the model call and the pipe, and the route stores no output for that turn.

The GET handler returns the stored items and their serial to the page.

// Agent code. This runs on your own infrastructure, so it holds the API key.
import OpenAI from 'openai';
import { openaiAdaptor } from '@ably/ai-transport/openai';
import { createRealtimeClient } from '@ably/pubsub-server';
import { loadConversation, recordItems } from '../../store';

const ably = createRealtimeClient({ key: process.env.ABLY_API_KEY });
const client = new OpenAI();

// Pass each event to the pipe, and keep the finished output items for your database.
async function* keepOutput(stream, onOutput) {
  for await (const event of stream) {
    if (event.type === 'response.completed') onOutput(event.response.output);
    yield event;
  }
}

export async function POST(req) {
  const { channelName, items } = await req.json();
  const { items: history } = loadConversation(channelName);

  const adaptor = openaiAdaptor(ably.channels.get(channelName));

  // Publish the user's input to the channel, and store it with its serial.
  const { serial: inputSerial } = await adaptor.send({ type: 'input', items });
  recordItems(channelName, items, inputSerial);

  try {
    const stream = await client.responses.create(
      { model: 'gpt-5.5', input: [...history, ...items], stream: true },
      { signal: req.signal },
    );

    let output = [];
    const { serial } = await adaptor.pipe(keepOutput(stream, (finished) => (output = finished)), {
      signal: req.signal,
    });

    recordItems(channelName, output, serial);
    return new Response(null, { status: 204 });
  } catch (error) {
    // The client closed the request, so the model call or the pipe was cancelled.
    if (req.signal.aborted) return new Response(null, { status: 204 });
    throw error;
  }
}

export function GET(req) {
  const channelName = new URL(req.url).searchParams.get('channel');
  return Response.json(loadConversation(channelName));
}

The codec sends the text that streamed as deltas once, so the response.completed event on the channel carries an empty output. The agent reads the output from the stream before the codec encodes it, so the items it stores keep their text.

Create the chat component

Create app/chat.js. The component reads the stored conversation from the agent's HTTP endpoint, and gets the channel with a rewind to its serial. It subscribes an openaiDecoder() to the channel. When the channel attaches, Ably replays the messages that arrived after the serial, in channel order, then delivers live messages. With no stored serial, nothing is stored yet, so the channel attaches without a rewind.

To send, the component wakes the agent with a POST that carries the user's input items. The agent reads the prompt from the request body, and publishes it to the channel, so every subscribed client receives it.

'use client';

// Client code. The browser fetches a token, so it never holds an API key.
import { useCallback, useEffect, useState } from 'react';
import { openaiDecoder } from '@ably/ai-transport/openai';
import { usePubSubClient } from '@ably/pubsub-device/react';
import { mergeEvent, toEntries } from './merge';

export function Chat({ channelName }) {
  const client = usePubSubClient();
  const [entries, setEntries] = useState([]);
  const [input, setInput] = useState('');

  const apply = useCallback(({ event }) => {
    if (event !== undefined) setEntries((prev) => mergeEvent(prev, event));
  }, []);

  useEffect(() => {
    let channel;
    let decoder;
    let cancelled = false;

    (async () => {
      const res = await fetch(`/api/chat?channel=${encodeURIComponent(channelName)}`);
      const stored = await res.json();
      if (cancelled) return;
      setEntries(toEntries(stored.items));
      // Replay the messages after the stored serial. With no stored serial, nothing is stored yet.
      channel = client.channels.get(channelName, stored.serial ? { params: { rewind: stored.serial } } : undefined);
      decoder = openaiDecoder();
      decoder.onEvent(apply);
      await channel.subscribe(decoder.receive);
    })();

    return () => {
      cancelled = true;
      if (!decoder) return;
      channel.unsubscribe(decoder.receive);
      // Detach, so the channel rewinds again when the effect runs again.
      channel.detach().catch(() => {});
    };
  }, [client, channelName, apply]);

  const send = async (e) => {
    e.preventDefault();
    const text = input.trim();
    if (!text) return;
    setInput('');
    const items = [{ role: 'user', content: text }];
    await fetch('/api/chat', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ channelName, items }) });
  };

  return (
    <div>
      {entries.map((entry) => (
        <p key={entry.id}>
          <strong>{entry.role}:</strong> {entry.text}
        </p>
      ))}
      <form onSubmit={send}>
        <input value={input} onChange={(e) => setInput(e.target.value)} />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

Merge the model output into conversation messages

Create app/merge.js. The agent publishes the Responses API's own stream events, and each client merges them into the entries its UI shows. Each response.output_text.delta appends its text to the entry for its output item, keyed by item_id, so a response builds up one entry as its deltas arrive. toEntries turns items into the same entries, for the stored items when the page loads and for the input items of each input event.

// Client code.
const textOf = (content) =>
  typeof content === 'string' ? content : content.map((part) => part.text ?? '').join('');

export function toEntries(items) {
  return items
    .filter((item) => item.role === 'user' || item.role === 'assistant')
    .map((item) => ({ id: item.id ?? crypto.randomUUID(), role: item.role, text: textOf(item.content) }));
}

export function mergeEvent(entries, event) {
  if (event.type === 'input') return [...entries, ...toEntries(event.items)];
  if (event.type !== 'response.output_text.delta') return entries;
  if (!entries.some((entry) => entry.id === event.item_id)) {
    return [...entries, { id: event.item_id, role: 'assistant', text: event.delta }];
  }
  return entries.map((entry) => (entry.id === event.item_id ? { ...entry, text: entry.text + event.delta } : entry));
}

Integrate with your database

Create app/store.js. This Map stands in for the database your application already uses. The Responses API takes its history as an input array of items, so the store keeps items, and the next turn passes them to the model unchanged. It also keeps the serial of the last stored input or response.

// Stand-in for your own database. Replace both functions with real queries.
const conversations = new Map();

export function loadConversation(channelName) {
  return conversations.get(channelName) ?? { items: [] };
}

export function recordItems(channelName, items, serial) {
  const current = loadConversation(channelName);
  conversations.set(channelName, { items: [...current.items, ...items], serial: serial ?? current.serial });
}

Wire it together

Create app/page.js. The page creates the Ably client in the browser with createClient, and wraps the chat in PubSubClientProvider:

'use client';

// Client code.
import { useEffect, useState } from 'react';
import { createClient } from '@ably/pubsub-device';
import { PubSubClientProvider } from '@ably/pubsub-device/react';
import { Chat } from './chat';

const channelName = 'conversations:demo';

export default function Page() {
  const [client, setClient] = useState();

  useEffect(() => {
    const ably = createClient({ authUrl: '/api/auth/token' });
    setClient(ably);
    return () => ably.close();
  }, []);

  if (!client) return null;

  return (
    <PubSubClientProvider client={client}>
      <Chat channelName={channelName} />
    </PubSubClientProvider>
  );
}

Create app/layout.js. Next.js needs a root layout to build the app:

export default function RootLayout({ children }) {
  return (
    <html lang="en">
      <body>{children}</body>
    </html>
  );
}

Run the app

Start the dev server:

npm run dev

Open http://localhost:3000 in two tabs and send a message from one. The response streams into both. The agent publishes the prompt to the channel, so both tabs show it. Reload a tab while a response is streaming, and the tab shows the stored conversation, then the response so far, then the rest of it live.

What happens when you send a message

1. The client POSTs the user's input items to the agent's HTTP endpoint.
2. The POST wakes the agent, and the agent reads the prompt from the request body. AI Transport does not send HTTP requests of its own. The agent publishes the prompt to the channel with send, and stores the input items with the serial that send returns.
3. The agent calls the Responses API with the stored items and the new input, and pipes the stream to the channel. Every subscribed client merges the deltas into the response.
4. When the pipe resolves, the agent stores the output items with the serial of the last message. A client that loads the page later replays the channel from that serial.

Understand the architecture

The OpenAI SDK owns the model call and the event stream. AI Transport carries that stream over one Ably channel, and the channel replays from the serial your database holds. Conversation covers who implements what.

Explore next

• History and replay: replay the channel from a stored serial, and read history backwards.
• Reconnection and recovery: what happens when a client loses its connection.
• OpenAI Responses: how the codec carries the Responses API's events.
• OpenAI API reference: openaiAdaptor, openaiDecoder, and the events the codec encodes and decodes.
• Agent adaptor API reference: every method on the adaptor.