Agent progress

Agent progress can show your users the work the agent is doing, either as a single live updating value, or as a plan that the agent is working through. There are different Ably channel features for the different ways you can expose agent progress to a client.

Pick how to show progress

LiveObjects allows you to create structured durable state on a channel. This is useful for modelling tasks or steps in a plan and ticking them off as the agent progresses. Ephemeral messages are like regular messages but they do not pollute conversation history, they are only delivered to live subscribers. This makes ephemeral messages useful for one-off status updates that provide useful progress updates to any client who is watching, but when those status updates are just noise to a late joining client. Presence tracks the state of agents and clients as present or absent, along with a status. Presence is useful if you need automatic updates to the state of a client even if that client does not gracefully disconnect.

The agent can use more than one option, such as a plan in LiveObjects and an ephemeral status for each tool call.

Show the plan with LiveObjects

An agent's plan might consist of some steps, the state of each step (planned, running, done, or failed) and the current step. The agent could write that plan to LiveObjects on the channel, so a client that attaches partway through a response receives the whole plan as it is now. In this example, the agent keeps the plan as one JSON value under a plan key.

The agent writes the plan with an HTTP client from createHttpClient, which needs the LiveObjects plugin from @ably/pubsub-server/liveobjects. The agent's API key needs the object-publish capability.

This agent code writes the plan as each step starts, and again when the last step finishes:

// channelName is the conversation's channel; doStep is your code that carries out one step.
import { createHttpClient } from '@ably/pubsub-server';
import { LiveObjects } from '@ably/pubsub-server/liveobjects';

const httpClient = createHttpClient({ key: process.env.ABLY_API_KEY, plugins: { LiveObjects } });
const object = httpClient.channels.get(channelName).object;

const plan = {
  state: 'running',
  current: 0,
  steps: [
    { title: 'Search flights', state: 'planned' },
    { title: 'Compare prices', state: 'planned' },
    { title: 'Book the cheapest flight', state: 'planned' },
  ],
};
const writePlan = () =>
  object.publish({ path: '', mapSet: { key: 'plan', value: { json: { ...plan, updatedAt: Date.now() } } } });

for (const [index, step] of plan.steps.entries()) {
  plan.current = index;
  step.state = 'running';
  await writePlan();
  await doStep(step);
  step.state = 'done';
}
plan.state = 'done';
await writePlan();

mapSet is one of the LiveObjects operations that the HTTP client publishes. With an empty path, it sets the plan key on the channel object, and the new value replaces the old value.

This client code shows the plan and marks the current step:

// The chat component's ChannelProvider wraps AgentPlan.
import { useObject } from '@ably/pubsub-device/liveobjects/react';

function AgentPlan() {
  const { value: plan, error } = useObject((object) => object.get('plan'));
  if (error || !plan) return null;
  return (
    <ol>
      {plan.steps.map((step, index) => (
        <li key={step.title} aria-current={index === plan.current ? 'step' : undefined}>
          {step.title}: {step.state}
        </li>
      ))}
    </ol>
  );
}

Without React, the client gets the channel object and subscribes to the plan key. object.get() attaches the channel, so the client should call it after the decoder subscribes. In this client code, channel is the client's channel, and renderPlan is your code that updates the UI:

const plan = (await channel.object.get()).get('plan');
renderPlan(plan.value());
plan.subscribe(() => renderPlan(plan.value()));

Send the status as ephemeral messages

An ephemeral message reaches every client subscribed to the channel at the moment the agent publishes it. Ably does not store it, so it stays out of history, rewind, and resume.

With the Vercel AI SDK, the agent can send a data-* part with transient: true through adaptor.send, and the Vercel adaptor publishes the part as an ephemeral message.

With any adaptor, the agent can publish a separate message to the same channel with extras: { ephemeral: true }.

In a custom codec, a row's encode can return ephemeral: true, and the adaptor then publishes that event as an ephemeral message.

pipeEach yields back to the agent after each event has been published to Ably, so the agent can send a status update when a tool call starts. The adaptor reads the next event from the LLM event stream only when the loop body finishes, so the agent should not await a slow publish call inside the loop.

In this agent code, adaptor is the agent adaptor, req is the request to your agent's HTTP endpoint and headers holds the headers that the agent sets on the response's messages, such as requestId. The code sends a status update each time the model calls a tool:

// stream is the OpenAI Responses stream; channel is the channel that the adaptor was created with.
for await (const { event } of adaptor.pipeEach(stream, { signal: req.signal, headers })) {
  if (event.type === 'response.output_item.added' && event.item.type === 'function_call') {
    channel
      .publish({ name: 'progress', data: { text: `Calling ${event.item.name}` }, extras: { ephemeral: true, headers } })
      .catch(console.error);
  }
}

This client code passes the latest status to setStatus, your code that updates the UI, and every other event to merge, your code that merges the response:

decoder.onEvent(({ event, message }) => {
  if (event === undefined && message.name === 'progress') {
    setStatus(message.data.text);
    return;
  }
  if (event !== undefined) merge(event, message.extras?.headers?.requestId);
});

Put the status in presence data

Each member of the presence set has a small data object, and a client receives the current data of every member when its channel attaches.

The agent should get its channel with the modes ['PUBLISH', 'PRESENCE'], so it does not receive the channel's messages or presence events. It can then enter presence once for each request and update its data at each step.

Ably replaces the presence data whole on each update, so each update should carry every field the status line needs, such as a status, the current step, the number of steps, and an updatedAt time.

This agent code updates its status between tool calls:

// The agent entered presence as agentId on a channel with ['PUBLISH', 'PRESENCE'].
await channel.presence.updateClient(agentId, {
  status: 'Searching flights',
  step: 2,
  of: 5,
  updatedAt: Date.now(),
});

The client can read the agent's status from the presence set with usePresenceListener in React, or with presence.subscribe.

When the agent stops

A LiveObjects plan stays as the agent last wrote it. If the agent crashes partway through the response, the plan remains as it was last written and does not change. To handle a plan that the agent no longer updates, you can combine any of these:

• The agent can mark the plan stopped, and the current step failed, in a finally block. The block runs after an error or a cancelled request, and does not run if the agent's process crashes.
• The client can treat a plan that says running as stale when its updatedAt time is older than the longest time your agent goes between updates.
• The agent's next response can write a new plan over the old one.

This agent code marks the plan when the agent stops without finishing it:

// doPlan works through the steps and calls writePlan, as in the example above.
try {
  await doPlan(plan);
} finally {
  if (plan.state === 'running') {
    plan.state = 'stopped';
    if (plan.steps[plan.current].state === 'running') plan.steps[plan.current].state = 'failed';
    await writePlan().catch(console.error);
  }
}

For Presence, Ably automatically removes the agent's and client's presence entry after being disconnected for about 15 seconds, or immediately if they call leaveClient. An agent that stops responding while its connection stays open stays in the presence set. The client can find an agent that stopped from an updatedAt time in the presence data.

Edge cases

• Two requests on one channel at the same time write to the same plan key and overwrite each other's plan. You can keep each request's plan under a separate key, such as plan:<requestId>, or allow one request at a time on a channel.
• An ephemeral status update and the response's text can arrive out of order. Ably holds the response's text in the rollup window and does not hold a plain or ephemeral message, so a status update can reach clients ahead of text the agent published before it. You should not use a status update to mark a point in the response's text.
• Ephemeral status updates that the agent publishes with its realtime client count against the connection's inbound message rate, along with the response's messages.
• Each write of a plan stored as one JSON value replaces the whole plan, so the last write wins. If the user also edits the plan, you can store the plan as a nested map, with one key for each step.
• Each write is one message, so a plan written as one value must fit within the maximum message size for your account. A plan with long step descriptions can keep them under separate keys.
• By default, the HTTP client's object.get() returns a JSON value as a JSON-encoded string. An agent that reads its plan back over HTTP has to parse the string, or read with compact: false, which decodes JSON values.
• LiveObjects state expires after the channel's object retention period passes with no update, and the channel object is then empty. If you need the final plan later, you should save it in your database.
• A client without the LiveObjects plugin, or without the OBJECT_SUBSCRIBE mode, gets an error from useObject, and the hook does not throw. The client can show the status line without the plan.

FAQ

Which option should I start with?

You should start with a LiveObjects plan, and use ephemeral messages for short-lived status updates. The agent uses both without attaching its channel, so it does not receive the channel's messages.

Does the plan end up in the conversation?

No. The agent writes the plan to LiveObjects on the channel, and the conversation's history does not include it. You can save the final plan in your database with the response if users need it later.

Can the client read progress after a reload?

Yes for a LiveObjects plan and presence data, because the client receives both when its channel attaches. The client receives an ephemeral status only with the next update that the agent sends.

Do OpenAI tool events already show progress?

Partly. The OpenAI adaptor publishes events such as response.web_search_call.searching as stored messages in the response, and they arrive at the client's onEvent handler. These events show only what the model is doing, and they stay in the conversation's history. To show the agent's plan and its steps, the agent can write them to LiveObjects.

Related features

• Presence: show the agent's status and who is on the channel.
• Ephemeral messages: send signals that reach every subscribed client and stay out of history.
• Live artifact state: keep artifact state in LiveObjects on the channel.