Completion and cancellation

AI responses streamed using the message-per-response or message-per-token pattern do not require explicit completion signals to function. Subscribers receive tokens as they arrive and can render them progressively. However, some applications benefit from explicitly signalling when a response is complete, or allowing users to cancel an in-progress response.

Benefits of completion and cancellation signals

Explicit completion and cancellation signals are useful when your application needs to:

• Detect whether a response is still in progress after reconnection, so clients can distinguish between a completed response and one that is still streaming
• Finalize UI state when a response ends, such as removing typing indicators or enabling input controls
• Allow users to abort a response mid-stream, stopping generation and saving compute resources
• Coordinate multiple content parts within a single response, where downstream logic depends on knowing when each part is finished

Signal completion

Use operation metadata to signal that a content part or response is complete. Operation metadata is a set of key-value pairs carried on each append or update operation. Subscribers can inspect this metadata to determine the current phase of a message.

Content-part completion

When streaming content using the message-per-response pattern, signal that a content part is complete by appending an empty string with a metadata marker. The empty append does not change the message's data, but the metadata signals to subscribers that no more content follows for this message.

This keeps the entire content lifecycle (create, stream, done) within a single Ably message:

const channel = realtime.channels.get('ai:big-bad-any');

// Publish initial message
const { serials: [msgSerial] } = await channel.publish({ name: 'response', data: '' });

// Stream tokens
for await (const event of stream) {
  if (event.type === 'token') {
    channel.appendMessage({
      serial: msgSerial,
      data: event.text
    }, {
      metadata: { phase: 'streaming' }
    });
  }
}

// Signal content-part completion with an empty append
channel.appendMessage({
  serial: msgSerial,
  data: ''
}, {
  metadata: { phase: 'done' }
});

Response-level completion

A single AI response may span multiple content parts, each represented as a separate Ably message with its own stream of appends. To signal that the entire response is complete, publish a discrete message after all content parts are finished. Subscribers can use this as a cue to finalize the response in the UI.

// After all content parts are done, signal response-level completion
await channel.publish({
  name: 'response-end',
  data: '',
  extras: {
    headers: {
      responseId: 'resp_abc123'
    }
  }
});

Detect completion from history

When hydrating client state from history, inspect version.metadata on each message to determine whether a content part was fully completed or is still in progress. If the most recent operation's metadata carries your completion marker, the content part is done. If it carries a streaming marker or no marker, the stream may still be active.

const channel = realtime.channels.get('ai:big-bad-any');

await channel.subscribe((message) => {
  // ...handle message actions as normal...
});

let page = await channel.history({ untilAttach: true });

while (page) {
  for (const message of page.items) {
    const phase = message.version?.metadata?.phase;

    if (phase === 'done') {
      // Content part is complete, render as final
    } else {
      // Content part may still be streaming, listen for live appends
    }
  }
  page = page.hasNext() ? await page.next() : null;
}

Cancel a response

Cancellation allows users to stop an in-progress response. The subscriber publishes a cancellation message on the channel, and the publisher stops generating and flushes any pending operations.

How it works

1. The subscriber publishes a cancellation message on the channel with a response ID identifying the response to cancel.
2. The publisher receives the cancellation message, stops generating, and flushes any pending append operations.
3. The publisher optionally publishes a confirmation message to signal clean shutdown to other subscribers.

Publish a cancellation request

The subscriber sends a cancellation message with a responseId in the message extras to identify which response to cancel:

const channel = realtime.channels.get('ai:big-bad-any');

// Send cancellation request for a specific response
await channel.publish({
  name: 'cancel',
  data: '',
  extras: {
    headers: {
      responseId: 'resp_abc123'
    }
  }
});

Handle cancellation

The publisher subscribes for cancellation messages and stops generation when one arrives. After stopping, flush any pending append operations before optionally publishing a confirmation message:

const channel = realtime.channels.get('ai:big-bad-any');

// Track pending appends for flushing
const pendingAppends = [];

// Listen for cancellation requests
await channel.subscribe('cancel', async (message) => {
  const responseId = message.extras?.headers?.responseId;

  // Stop generation for the matching response
  stopGeneration(responseId);

  // Flush any pending appends before confirming
  await Promise.all(pendingAppends);

  // Optionally confirm cancellation to all subscribers
  await channel.publish({
    name: 'cancelled',
    data: '',
    extras: {
      headers: {
        responseId
      }
    }
  });
});