Tool calling

Tool calling in AI Transport supports both server-executed and client-executed tools. Tool invocations and results are published to the session, so every client sees tool activity in realtime and tool state persists in history.

How it works

When the LLM invokes a tool, the invocation is streamed through the session like any other turn event. Clients see tool calls appear as they are generated. If the tool runs on the server, the result is streamed back in the same run. If the tool runs on the client, the agent calls run.suspend() so the run stays live, and the client submits the result on a continuation invocation. What the continuation does next depends on the codec: the Vercel codec forks the result into a new reply run, and the OpenAI codec resumes the suspended run.

Tool state (invocations, arguments, results) is part of the session's history. Late joiners and reconnecting clients see the full tool activity as well as the final text.

Server-executed tools

Server-executed tools are the default path. On the agent, the AI SDK handles tool execution during the LLM stream. Tool invocations and results are encoded by the codec and published to the session as part of the turn.

const result = streamText({
  model: anthropic('claude-sonnet-4-20250514'),
  messages: conversationHistory,
  tools: {
    getWeather: {
      description: 'Get current weather for a location',
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => {
        const data = await fetchWeather(city);
        return { temperature: data.temp, conditions: data.conditions };
      },
    },
  },
  abortSignal: run.abortSignal,
});

const { reason } = await run.pipe(result.toUIMessageStream());
await run.end({ reason });

Clients see the tool invocation as it streams, then the result, then the LLM's follow-up text, all within a single turn.

Client-executed tools

Client-executed tools require a round trip between the server and the client. The LLM requests a tool call, the turn ends, the client executes the tool locally and submits the result, and a continuation turn starts.

On the server, define the tool without an execute function. When the LLM invokes it, the stream ends with a tool call that the client must fulfil:

const result = streamText({
  model: anthropic('claude-sonnet-4-20250514'),
  messages: conversationHistory,
  tools: {
    getUserLocation: {
      description: "Get the user's current location",
      inputSchema: z.object({}),
      // No execute function: the client handles this.
    },
  },
  abortSignal: run.abortSignal,
});

const pipeResult = await run.pipe(result.toUIMessageStream());
const outcome = await vercelRunOutcome(pipeResult, result.finishReason);

if (outcome.reason === 'suspend') {
  await run.suspend();
} else {
  await run.end(outcome);
}

vercelRunOutcome returns 'suspend' when streamText finishes with finishReason: 'tool-calls', so the agent suspends instead of ending. The pending tool call stays on the session for any connected client to fulfil.

On the client, find the assistant message with the pending tool call and publish the result addressed to it. With the Vercel codec the result opens a new reply run rather than re-entering the suspended one, and createToolResultFork builds both the input and the send options for that fork. The fork carries a copy of the suspended run's full message list, so the new run reconstructs the whole turn with the result folded in before the agent continues generating. Seeding the whole run rather than the single assistant message keeps context across sequential client tool calls.

A tool part arrives in one of two representations: a statically-declared tool (one defined in the tools object, like getUserLocation above) as tool-${name}, and a dynamic tool as dynamic-tool. Match both so the lookup works regardless of how the tool was declared:

// Client code.
import { createToolResultFork, createUIMessageCodec } from '@ably/ai-transport/vercel';

const codec = createUIMessageCodec();

const { messages, runOf, send } = useView();
const { getRunNode } = useTree();

const isToolPart = (p) => p.type === 'dynamic-tool' || p.type.startsWith('tool-');

const pending = messages.find(({ message }) =>
  message.parts?.some((p) => isToolPart(p) && p.state === 'input-available'),
);

if (pending) {
  const toolCall = pending.message.parts.find(
    (p) => isToolPart(p) && p.state === 'input-available',
  );

  const location = await new Promise((resolve, reject) => {
    navigator.geolocation.getCurrentPosition(resolve, reject);
  });

  // Read the suspended run from the tree rather than guessing from message order.
  const node = getRunNode(runOf(pending.codecMessageId).runId);

  const { input, sendOptions } = createToolResultFork({
    runMessages: codec.getMessages(node.projection),
    parentCodecMessageId: node.parentCodecMessageId,
    toolCallId: toolCall.toolCallId,
    result: { output: { lat: location.coords.latitude, lng: location.coords.longitude } },
    supersedesRunId: node.runId,
  });

  const forked = await send([input], sendOptions);

  // Wake the agent so it picks up the result and starts the fork's run.
  await fetch('/api/chat', {
    method: 'POST',
    body: JSON.stringify(forked.toInvocation().toJSON()),
  });
}

The fork is published without a run id. The agent mints the run id when it starts the reply run, and supersedesRunId marks the suspended run resolved so the tree hides it from branch selection. A single client answering once renders as one linear reply, and two clients answering the same tool call produce two forks that stay on segregated sibling branches.

With useChat, the chat transport performs this construction for you when the hook submits a tool result, so you never build the fork by hand.

OpenAI codec

The examples above use the Vercel codec, but the OpenAI Responses codec supports the same tools: server-executed function calls, client-executed tools, tool failures, and human approvals. Two things differ. The wire types and the factory payload field names are the Responses ones, and a client resolution resumes the suspended run instead of forking, so you pass the suspended run's runId in the send options.

The OpenAI codec expresses tool state against the Responses types, so a tool call is a function_call item and its result is a function_call_output item. The client factories take snake_case payloads keyed by call_id, and you address each to the assistant message that holds the call:

import { ResponsesCodec } from '@ably/ai-transport/openai';

// A client-run tool succeeded.
await view.send(ResponsesCodec.createToolResult(codecMessageId, { call_id, output }), { runId });

// A client-run tool failed. The message becomes the output the model sees next turn.
await view.send(ResponsesCodec.createToolResultError(codecMessageId, { call_id, message }), { runId });

// A user approved or denied a gated tool. A denial resolves entirely on the client.
await view.send(ResponsesCodec.createToolApprovalResponse(codecMessageId, { call_id, approved, reason }), { runId });

The Responses function_call_output item has no field for an approval decision or an error, so the codec holds that render-only state on OpenAIMessage.toolCallStates, a map keyed by call_id. toResponsesInput never reads it, so it cannot reach the model. See OpenAI Responses for the agentic loop and the approval-request output.

History persistence

Tool invocations and results are part of the session's history. When a client reconnects or a late joiner loads the conversation, tool activity is replayed along with text messages. The view reconstructs tool state so the UI shows the correct status: pending, complete, or failed.

A user who starts a tool-assisted workflow on a laptop continues it on a phone without losing context.

Durable tool execution

When the agent runs inside a workflow engine such as Temporal or Vercel WDK, each tool execution can be its own retryable activity. Wrap the tool call in AgentRun.createStep({ stepId }) and publish the result via RunStep.send. A retry of the same tool activity re-enters createStep with the same stepId, so the retry's tool result supersedes the failed attempt on the session rather than appending beside it.

Edge cases and unhappy paths

• A client-executed tool that the user denies (for example a geolocation permission prompt) leaves the tool call pending. Submit a failure to unblock the LLM, or end the turn explicitly. On the Vercel path pass result: { errorMessage } to createToolResultFork; on the OpenAI path use ResponsesCodec.createToolResultError.
• A tool that takes longer than the agent's runtime budget suspends the run rather than ending it. When the result is ready, publish it as a continuation addressed to the original message using the pattern shown above. Do not publish a plain new turn to deliver a late result, because the agent then has no suspended run to resolve.
• A server-executed tool that does not honour run.abortSignal keeps running after a cancel. Wire the signal into your tool implementation.
• Two clients submitting the same client-executed tool concurrently behave differently per codec. With the Vercel codec each submission forks, so the two answers land on segregated sibling branches and need no application-level guard. With the OpenAI codec both resolutions fold into the one suspended run, keyed by call_id on a last-write-wins basis, so guard against double-submit at the application layer.
• A failed tool call is delivered with an error result. The view exposes the failure; render it in place rather than silently retrying.

FAQ

Do server-executed and client-executed tools mix in one turn?

Yes. The LLM may invoke any tool the agent defines. Server-executed tools complete inline; client-executed tools end the turn and resume in a continuation turn.

How do I cancel a tool call?

Cancel the turn. The agent's abortSignal fires; if your tool implementation checks it, the tool stops. Pending client-executed tools do not invoke if the turn is cancelled before submission.

What if my client cannot perform the tool?

Submit a tool result with an error payload. The agent receives it on the continuation turn and decides how to respond.

Are tool inputs and outputs visible to every participant?

Yes. Tool calls are messages on the session, so every subscriber sees them. Scope channel capabilities if you need to restrict visibility.

How big can a tool result be?

Subject to Ably's message size limit. Stream large results across multiple events or persist them externally and reference the URL.

Related features

• Human-in-the-loop: approval gates built on tool calling.
• Token streaming: how tool events are streamed.
• History and replay: loading past tool activity from history.
• Durable execution: run each tool as its own retryable step under a workflow engine.