Live artifact state

This page uses the example of an agent that builds a table while it researches, and writes one row to LiveObjects for each result it finds. By following through the example, you can see how LiveObjects can be used to represent different types of artifact state, such as tables, file trees, plans and task lists, tickets and Kanban boards, work queues and more.

How it works

The table lives in LiveObjects on the same Ably channel as the conversation. The table key of the channel object holds a map from each row's id to the row.

The agent writes rows over HTTP with an HTTP client. The HTTP client does not attach the channel, so the agent does not receive the channel's messages. Each user's client gets the channel with the object channel modes. When the client's channel attaches, including after a reload or when the user joins late, the client receives the table as it is now, and then each change as the agent or a user makes it.

Your database keeps the durable copy of the table, and the conversation keeps a reference to it, such as the table's id.

Enable LiveObjects

Object operations need each of these three requirements in place:

This agent code creates the HTTP client with the plugin:

import { createHttpClient } from '@ably/pubsub-server';
import { LiveObjects } from '@ably/pubsub-server/liveobjects';

const httpClient = createHttpClient({ key: process.env.ABLY_API_KEY, plugins: { LiveObjects } });

This client code gets the channel with the modes, and with the rewind param that loads the conversation from the serial in your database:

// The browser fetches a token, so it never holds an API key.
import { createClient } from '@ably/pubsub-device';
import { LiveObjects } from '@ably/pubsub-device/liveobjects';

const modes = ['MESSAGE_SUBSCRIBE', 'OBJECT_SUBSCRIBE', 'OBJECT_PUBLISH'];

const client = createClient({ authUrl: '/auth', plugins: { LiveObjects } });
const channel = client.channels.get(channelName, storedSerial ? { modes, params: { rewind: storedSerial } } : { modes });

In React, you pass the modes in the options of ChannelProvider, together with the rewind param. This client code shows the chat component in a page:

// The modes array is defined outside the component, so it stays the same on every render.
import { createClient } from '@ably/pubsub-device';
import { LiveObjects } from '@ably/pubsub-device/liveobjects';
import { ChannelProvider, PubSubClientProvider } from '@ably/pubsub-device/react';

const modes = ['MESSAGE_SUBSCRIBE', 'OBJECT_SUBSCRIBE', 'OBJECT_PUBLISH'];
const client = createClient({ authUrl: '/auth', plugins: { LiveObjects } });

function Page({ channelName, stored }) {
  const options = stored.serial ? { modes, params: { rewind: stored.serial } } : { modes };
  return (
    <PubSubClientProvider client={client}>
      <ChannelProvider channelName={channelName} options={options}>
        <Chat channelName={channelName} stored={stored.messages} />
      </ChannelProvider>
    </PubSubClientProvider>
  );
}

Write the artifact from the agent

The agent writes the table over HTTP with object.publish. It creates the table with mapCreate at path: 'table', which creates an empty map and assigns it to the table key of the channel object in one request. A mapCreate at a key that already holds a table replaces that table and its rows, so the agent creates the table only when the channel object does not have a table key.

The agent then sets one key for each row with mapSet at path: 'table'. The value { json: row } stores the row as one JSON value, so the agent writes each row in one operation. Each row carries a createdAt time, so the client can sort the rows.

This agent code creates the table and writes each row as the agent finds it:

import { createHttpClient } from '@ably/pubsub-server';
import { LiveObjects } from '@ably/pubsub-server/liveobjects';

const httpClient = createHttpClient({ key: process.env.ABLY_API_KEY, plugins: { LiveObjects } });
const object = httpClient.channels.get(channelName).object;

// Create the table when the channel object does not have a table yet.
const channelObject = await object.get();
if (channelObject?.table === undefined) {
  await object.publish({ path: 'table', mapCreate: { semantics: 'lww', entries: {} } });
}

// Call writeRow each time the agent finds a result.
async function writeRow(result) {
  const row = { id: crypto.randomUUID(), createdAt: Date.now(), ...result };
  await object.publish({ path: 'table', mapSet: { key: row.id, value: { json: row } } });
}

A write to a row stored as one JSON value replaces the whole row. A row stored as a nested map lets the agent and a user change different cells of one row, and each cell is its own operation.

The agent can also write the table with its realtime client. The agent needs the realtime client to react to the user's edits as they happen. object.get() on a realtime channel attaches the channel. With the modes PUBLISH, OBJECT_SUBSCRIBE, and OBJECT_PUBLISH, the agent receives object updates and does not receive messages. The agent must pass the modes on its first channels.get for the channel, and give the adaptor the same channel. The table must exist before the agent sets a row, because set on a path that does not resolve to a map throws.

This agent code writes a row with the realtime client:

import { createRealtimeClient } from '@ably/pubsub-server';
import { LiveMap, LiveObjects } from '@ably/pubsub-server/liveobjects';
import { vercelAdaptor } from '@ably/ai-transport/vercel';

const ably = createRealtimeClient({ key: process.env.ABLY_API_KEY, plugins: { LiveObjects } });
const channel = ably.channels.get(channelName, { modes: ['PUBLISH', 'OBJECT_SUBSCRIBE', 'OBJECT_PUBLISH'] });
const adaptor = vercelAdaptor(channel);

const object = await channel.object.get();
if (object.get('table').compact() === undefined) await object.set('table', LiveMap.create({}));
await object.get('table').set(row.id, row);

Show the artifact

In React, useObject from @ably/pubsub-device/liveobjects/react subscribes to the table key and updates the component on each change. value is undefined until the channel object syncs, and while the channel object does not have a table. error carries the reason when the object cannot load, such as a missing plugin or mode.

useObject attaches the channel itself. You should call the hook in the chat component or in a component inside it, so the hook mounts and unmounts with the decoder's subscription. A component that mounts before the chat component can attach the channel before the decoder subscribes, and the decoder then misses the messages that the rewind delivers.

A map does not keep the order of its keys, so the client should sort the rows by a field in each row, such as createdAt.

This client code shows the table:

// The component runs under the chat component's ChannelProvider,
// and the channel has the object modes.
import { useObject } from '@ably/pubsub-device/liveobjects/react';

function ResultsTable() {
  const { value: table, error } = useObject((object) => object.get('table'));
  if (error || !table) return null;
  const rows = Object.values(table).sort((a, b) => a.createdAt - b.createdAt);
  return (
    <table>
      <tbody>
        {rows.map((row) => (
          <tr key={row.id}>
            <td>{row.title}</td>
            <td>{row.summary}</td>
          </tr>
        ))}
      </tbody>
    </table>
  );
}

Without React, the client gets the channel object and subscribes to the table key. object.get() attaches the channel, so the client should call it after the decoder subscribes:

// The client has the LiveObjects plugin and the channel has the object modes.
const table = (await channel.object.get()).get('table');
const render = () => renderRows(Object.values(table.compact() ?? {}).sort((a, b) => a.createdAt - b.createdAt));
render();
table.subscribe(render);

Store a durable copy

You are responsible for keeping the durable copy of the table in your database. LiveObjects keeps the table on the channel only until the channel's object retention period passes with no update, and the channel object is then empty.

The agent can write the table to your database after each tool call, or when it finishes building the table. When the agent reads the table back from the channel, it gets the user's edits as well as its own rows. The HTTP client's compact object.get() returns each JSON row as a JSON-encoded string, so the agent has to parse each row. A row stored as a map comes back as an object.

This agent code reads the table from the channel and saves it to your database:

// saveArtifact writes to your database.
const table = await httpClient.channels.get(channelName).object.get({ path: 'table' });
const rows = Object.values(table ?? {}).map((row) => JSON.parse(row));
await saveArtifact(channelName, rows);

The conversation keeps a reference to the table. With the Vercel adaptor, the reference can be a data-artifact part in the response, which Ably stores on the channel with the rest of the response. With other adaptors, the agent can publish its own message on the channel with the table's id.

When a client loads the conversation, it can read the table from your database with the conversation, and show the stored rows until the channel object syncs. In React, the component can show the stored rows while value is undefined, and the rows from value after that.

If the object retention period passed with no update, the channel object does not have a table. The agent can restore the table from your database with one mapCreate that carries the stored rows as its entries, before it writes again. This agent code restores the table, in place of the mapCreate with empty entries in Write the artifact from the agent:

// loadArtifact reads the rows from your database.
const channelObject = await object.get();
if (channelObject?.table === undefined) {
  const rows = await loadArtifact(channelName);
  const entries = Object.fromEntries(rows.map((row) => [row.id, { data: { json: row } }]));
  await object.publish({ path: 'table', mapCreate: { semantics: 'lww', entries } });
}

Let users edit the artifact

The client writes the table with the same objects API that it reads with. The client's channel needs the OBJECT_PUBLISH mode, and the token needs the object-publish capability.

A row stored as one JSON value changes only as a whole, so the client must write the whole row with table.set(rowId, row). If the agent writes the same row at the same time, one of the two writes is lost. To let the user and the agent change different cells of one row, the agent can store each row as a map, with one key for each cell.

This agent code creates a row as a map:

// mapCreate at table.<rowId> creates the row's map and assigns it to the table.
await object.publish({
  path: `table.${row.id}`,
  mapCreate: {
    semantics: 'lww',
    entries: {
      title: { data: { string: row.title } },
      createdAt: { data: { number: row.createdAt } },
    },
  },
});

This client code edits one cell of the row:

// useObject returns the table's live object as well as its value.
const { object: table } = useObject((object) => object.get('table'));
const saveNotes = (rowId, notes) => table.get(rowId).set('notes', notes);

When two writers change one key at the same time, the result depends on the operation: two increments of a LiveCounter both take effect, and a map set on that key is last-write-wins. Concurrent writes to different keys all apply, so you can partition the writes by key, with the agent writing the cells it fills in, such as title and summary, and the user writing their own cells, such as notes.

The agent sees the user's edits the next time it reads the table over HTTP with object.get({ path: 'table' }). To react to an edit as it happens, the agent can subscribe to the table key with its realtime client.

Edge cases

• A client without the LiveObjects plugin gets an error from useObject, and channel.object throws in plain JavaScript. Create the client with plugins: { LiveObjects }.
• A channel without the OBJECT_SUBSCRIBE mode makes object.get() throw a 40024 error, and a channel without OBJECT_PUBLISH makes a write throw a 40024 error. Pass the object modes with the default modes. Ably also drops a mode from the attached channel when the token lacks the matching capability, so a 40024 error after you set the modes means the token lacks object-subscribe or object-publish.
• A client that lists only the object modes stops receiving the agent's response. List MESSAGE_SUBSCRIBE as well.
• channels.get with other modes throws when the channel is already attached. Pass the modes on the first channels.get for the channel, or change them with channel.setOptions, which reattaches the channel.
• Changing the modes or the rewind param in the options of ChannelProvider after the channel attaches reattaches the channel. Keep the modes array the same for the life of the component.
• A client created with echoMessages: false cannot write objects, and the write throws. Leave echoMessages at its default on a client that edits the table.
• Ably resolves a path to an object when it receives the request. If a concurrent write replaced the table map, the operation applies to the old map, and the row is lost. Do not create the table again while the agent writes rows.
• A mapSet on a key that two writers change at the same time is last-write-wins, and one write is lost.
• The table expires after the channel's object retention period passes with no update, and the channel object is then empty. Restore it from your durable copy.
• All objects on a channel share an aggregate size limit, and each operation must fit in one message. If the agent restores a large table in one mapCreate, the operation can go over the size of one message, so create the empty table and restore the rows in several batches. Keep large cells, such as files and long text, in blob storage, and store a URL in the cell.
• A row id that contains a dot breaks a path such as table.<rowId>. Use ids without dots, or escape the dot with a backslash.

FAQ

What belongs in the artifact, and what in the conversation?

The artifact holds the current state that the user and the agent both act on, such as the table and its rows. The conversation holds the prompts and responses, and a reference to the artifact, such as the table's id.

Why do object operations throw?

One of the three requirements is missing: the LiveObjects plugin on the client, the object modes on the client's channel, or the object capabilities on the token or key.

Can the user and the agent both edit the table?

Yes. Both write with the objects API on the same channel. A map set on one key is last-write-wins, so you can partition the writes by key.

Are there React hooks for LiveObjects?

Yes. useObject from @ably/pubsub-device/liveobjects/react returns the value of the object you select, and updates the component on each change. It also returns the live object, which the component can use to write.

Does the agent need a realtime connection to write the table?

No. The agent's HTTP client writes the table over HTTP, and does not attach the channel. The agent needs its realtime client only to react to the user's edits as they happen.

Related features

• Artifacts: where an artifact lives, and what belongs in the conversation and what in state.
• Agent progress: show the agent's plan and progress with LiveObjects on the same channel.
• History and replay: load the stored conversation and catch up from the channel.
• LiveObjects: the LiveObjects API, including LiveMap and LiveCounter.