Redis pub/sub limitations for Temporal frontend delivery

A Temporal activity completes and publishes the result to a Redis channel. A relay service subscribes to that channel and forwards messages to SSE endpoints the browser holds open. The workflow doesn't know or care how events reach the client.

The pattern is described in detail by the team at Architecting Bytes, who built a production implementation. Their writeup covers the full relay architecture: activity publishes to Redis, a FastAPI or similar service subscribes and streams to the browser.

A minimal Temporal activity publishing to Redis looks like this:

from temporalio import activity
import redis.asyncio as redis

@activity.defn
async def stream_token_to_redis(channel: str, token: str) -> None:
    # Fire-and-forget: no delivery guarantee if no subscriber is connected
    r = redis.from_url("redis://localhost")
    await r.publish(channel, token)

For simple cases where clients stay connected throughout, this is a workable architecture. With generation and delivery decoupled, the relay can be scaled independently.

Redis pub/sub delivers only to currently connected subscribers. That limit is structural, not configurable. If no one is listening when a message arrives, it disappears.

The Architecting Bytes developer who shipped this architecture in production was direct: "This is not a durable delivery format, it is purely fire-n-forget."

The limitation is structural. Pub/sub has no storage layer by design: it is a realtime broadcast primitive with no consumer groups, acknowledgment mechanism, or message buffer. Messages that arrive when no subscriber is connected are discarded immediately.

Redis Streams is a different primitive that offers persistence and consumer groups. Moving to it solves the storage gap but changes the architecture: the relay must now poll or block-read from the stream rather than receive push events, and the consumer group model introduces its own offset management complexity. Teams typically reach for pub/sub first because it is simpler to set up. Most hit its limits in production before considering a move to Redis Streams.

• Client disconnects mid-generation. When a mobile network drops, a corporate proxy times out a long-lived SSE connection, or a tab closes and reopens, the Redis channel carries on receiving events from the Temporal activity with no subscriber. Every event during the disconnect is discarded. When the client reconnects, it picks up from the current moment. Everything that happened while they were gone is permanently lost.

• Multiple subscribers with divergent state. A second tab opened during an active generation starts receiving from that moment. Events that arrived before it connected are permanently missing. Two tabs watching the same workflow end up with different, incomplete views.

• Relay process restarts. The relay service between Redis and the SSE endpoint is infrastructure you own. When it restarts for a deploy, a crash, or a scaling event, all connected SSE clients are dropped. Clients that reconnect to the new relay instance start from the current event position, with no access to in-flight connections or event history.

• Workflow termination and the Kafka alternative. Some teams consider a Temporal-to-Kafka interceptor as an alternative to Redis pub/sub for more durable delivery. This approach has a documented limitation: the Temporal Kafka interceptor is not called in cases of workflow termination or timeout. Events from terminated workflows are silently dropped, making the pattern unreliable for anything that depends on workflow completion events.

For short workflows where users stay on a single device for the full run, Redis pub/sub handles the job. The fire-and-forget model is fine when a network drop or second device is not a realistic concern. Internal tooling, demo environments, and developer-facing dashboards often fit this profile.

The limit is that these conditions rarely hold in production-facing applications. Mobile networks drop, users open second tabs, and long-running agents often complete while users have moved to a different page. The longer the workflow runs, the more likely a disconnect.

Temporal's Workflow Streams improves on Redis pub/sub for a single connected device: subscribers track their own offset client-side, so a reconnecting client can resume from where it left off.

A second device joining mid-run has no offset to start from. It receives events from the moment it connects and has no way to retrieve what came before. Two devices watching the same workflow see different, incomplete views.

Workflow Streams is also Python-only. Per-roundtrip latency is around 100ms, which suits step-level progress updates better than token-by-token streaming.

A delivery layer built for agent workflows handles the cases Redis pub/sub and Workflow Streams can't. Events are buffered during client disconnects and replayed in order on reconnect. One publish from a Temporal activity reaches all subscribed clients.

Temporal continues handling execution. The delivery layer handles session state: what clients have received, what they missed, where each one is in the stream.

The result is execution durability and delivery durability working at different layers of the same stack.

Ably AI Transport provides a delivery layer that works alongside Temporal, replacing the Redis-to-SSE relay with durable, resumable streaming. Visit the Ably AI Transport overview, read the documentation, or sign up free to start building. 

Can Redis Streams solve the reconnect problem that Redis pub/sub has?

Redis Streams does offer persistence and consumer group offset management, which Redis pub/sub lacks. But it changes the architecture meaningfully: the relay must now poll or block-read from the stream rather than receive push events, and consumer group offset tracking adds operational complexity. Teams that migrate from pub/sub to Streams typically find they've traded one set of problems for another. The replay-on-reconnect problem - where a specific client needs to resume from exactly where it left off - requires per-client offset tracking that Redis Streams does not handle out of the box.

When is Redis pub/sub good enough for a Temporal frontend?

For short workflows where users stay on a single device and network drops are unlikely, Redis pub/sub is fine. Internal tooling, developer-facing dashboards, and demo environments often meet these conditions. The fire-and-forget model becomes a problem as workflow duration increases - the longer the workflow, the more likely a client disconnects at some point during execution.

Why does the Kafka interceptor not cover workflow termination events?

The Temporal Kafka interceptor hooks into workflow lifecycle events during normal execution. When a workflow is forcibly terminated or times out, Temporal does not call the interceptor. This means any downstream system relying on Kafka for terminal state notifications - "workflow complete," "workflow failed" - will miss those events silently. It is documented behaviour, not a bug, but it makes the pattern unreliable for use cases that depend on completion events reaching the frontend.