Tool calling

Tool calling in AI Transport supports both server-executed and client-executed tools. Tool invocations and results are published to the channel, so every client sees tool activity in real time and tool state persists in history.

How it works

When the LLM invokes a tool, the invocation is streamed through the channel like any other turn event. Clients see tool calls appear as they are generated. If the tool runs on the server, the result is streamed back in the same turn. If the tool runs on the client, the turn ends; the client submits the result, and a continuation turn starts.

Tool state (invocations, arguments, results) is part of the channel's message history. Late joiners and reconnecting clients see the full tool activity, not just the final text.

Server-executed tools

Server-executed tools are the default path. The AI SDK handles tool execution during the LLM stream. Tool invocations and results are encoded by the codec and published to the channel as part of the turn.

const result = streamText({
  model: anthropic('claude-sonnet-4-20250514'),
  messages: conversationHistory,
  tools: {
    getWeather: {
      description: 'Get current weather for a location',
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => {
        const data = await fetchWeather(city);
        return { temperature: data.temp, conditions: data.conditions };
      },
    },
  },
  abortSignal: turn.abortSignal,
});

const { reason } = await turn.streamResponse(result.toUIMessageStream());
await turn.end(reason);

Clients see the tool invocation as it streams, then the result, then the LLM's follow-up text, all within a single turn.

Client-executed tools

Client-executed tools require a round trip between the server and the client. The LLM requests a tool call, the turn ends, the client executes the tool locally and submits the result, and a continuation turn starts.

On the server, define the tool without an execute function. When the LLM invokes it, the stream ends with a tool call that the client must fulfil:

const result = streamText({
  model: anthropic('claude-sonnet-4-20250514'),
  messages: conversationHistory,
  tools: {
    getUserLocation: {
      description: "Get the user's current location",
      inputSchema: z.object({}),
      // No execute function: the client handles this.
    },
  },
  abortSignal: turn.abortSignal,
});

const { reason } = await turn.streamResponse(result.toUIMessageStream());
await turn.end(reason);

On the client, detect the pending tool call and submit the result using view.update(). The first argument is the Ably message ID of the node containing the tool invocation, not the tool call ID:

const { nodes } = useView({ transport });

const pendingNode = nodes.find((n) =>
  n.message.parts?.some((p) => p.type === 'dynamic-tool' && p.state === 'input-available'),
);

if (pendingNode) {
  // navigator.geolocation.getCurrentPosition is callback-based, not a promise.
  const location = await new Promise((resolve, reject) => {
    navigator.geolocation.getCurrentPosition(resolve, reject);
  });

  const toolCall = pendingNode.message.parts.find(
    (p) => p.type === 'dynamic-tool' && p.state === 'input-available',
  );

  await view.update(pendingNode.id, [{
    type: 'tool-output-available',
    toolCallId: toolCall.toolCallId,
    output: { lat: location.coords.latitude, lng: location.coords.longitude },
  }]);
}

Calling view.update() submits the tool result to the server and triggers a continuation turn. The server receives the result, includes it in the conversation history, and the LLM generates a response that incorporates the tool output.

Cross-turn events with EventsNode

addEvents() delivers events to an existing assistant message. A common use is delivering tool results to a message that contains a pending tool call. Target the message by its Ably message ID:

const assistantMsgId = pendingNode.id;
const toolCallId = pendingToolCall.toolCallId;

await turn.addEvents(assistantMsgId, [
  {
    type: 'tool-output-available',
    toolCallId,
    output: { temperature: 22, conditions: 'sunny' },
  },
]);

Events published through addEvents() update the target message in the view. Because they target an existing message by its ID, late joiners and reconnecting clients see the correct state when the conversation replays from history.

History persistence

Tool invocations and results are part of the channel's message history. When a client reconnects or a late joiner loads the conversation, tool activity is replayed along with text messages. The view reconstructs tool state so the UI shows the correct status: pending, complete, or failed.

A user who starts a tool-assisted workflow on a laptop continues it on a phone without losing context.

Edge cases and unhappy paths

• A client-executed tool that the user denies (for example a geolocation permission prompt) leaves the tool call pending. Submit a failure result to unblock the LLM, or end the turn explicitly.
• A tool that takes longer than the agent's runtime budget runs past turn.end(). The continuation turn picks up the result through addEvents on the original message; do not start a new turn just to deliver a late result.
• A server-executed tool that does not honour turn.abortSignal keeps running after a cancel. Wire the signal into your tool implementation.
• Two clients submitting the same client-executed tool concurrently produce two continuation turns. Guard against double-submit at the application layer.
• A failed tool call is delivered with an error result. The view exposes the failure; render it in place rather than silently retrying.

FAQ

Do server-executed and client-executed tools mix in one turn?

Yes. The LLM may invoke any tool the agent defines. Server-executed tools complete inline; client-executed tools end the turn and resume in a continuation turn.

How do I cancel a tool call?

Cancel the turn. The agent's abortSignal fires; if your tool implementation checks it, the tool stops. Pending client-executed tools do not invoke if the turn is cancelled before submission.

What if my client cannot perform the tool?

Submit a tool result with an error payload. The agent receives it on the continuation turn and decides how to respond.

Are tool inputs and outputs visible to every participant?

Yes. Tool calls are messages on the channel, so every subscriber sees them. Scope channel capabilities if you need to restrict visibility.

How big can a tool result be?

Subject to Ably's message size limit. See the platform limits. Stream large results across multiple events or persist them externally and reference the URL.

Related features

• Human-in-the-loop: approval gates built on tool calling.
• Token streaming: how tool events are streamed.
• History and replay: loading past tool activity from history.